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A considerable fraction of the numerous decisions re- 
quired in industry are made as intuitive judgments; they are 
"nunch" decisions, Such decisions are often made without any 
conscious attempt to define the alternatives that are availe 
able, Even though alternatives are defined, it is common for 
no effort to be made to express the prospective differences 

between the alternatives in commensurable units, 


In contrast, some decisions in industry are based on 
engineering economy studies, In an economy study, an attempt 
is made to discover all of the promising alternatives and to 
arrive at a clear definition of the alternatives to be come 
pared, Estimates are made of the prospective consequences of 
choosing each of the alternatives, In so far as possible, 
these expected consequences are made commensurable by being 
expressed in terms of prospective money disbursements and, 
whenever appropriate, prospective money receipts, Where proe 
posed investments are involved, the prospective money time 
series are made comparable by calculations that reflect the 
time value of money, Finally a choice among the alternatives 
is made that gives weight not only to their money differences 
but also to those prospective differences that have not been 
reduced to money terms, 


Many decisions in industry relate in some way to matters 


connected with quality, It is my impression that the techniques 
of engineering economy have been used less frequently in quality 


matters than in a number of other arease of decision making, 
No doubt one reason for this failure to make economy studies 
is the fact that decisions on quality matters are so numerous 
that it is felt that little time can be devoted to each decie 


sion, Another reason is the difficulty of placing a money val- 


uation on many of the differences between alternatives that 
require consideration in dealing with quakity matters, 


How can statistical quality control contribute to better 


decisions on matters related to the economic aspects of quality? 


This contribution may be made in two ways, One of these ways 
is through the provision of better alternatives than were pre- 
viously available in dealing with certain problems involving 
quality, Familiar examples of such better alternatives are 
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the use of the Shewhart control chart for controlling quality 
during production and the use of acceptance procedures such as 
are described in the Dodge-Romig tables and the Navy tables, 
The other way in which statistical quality control contributes 
to better decisions is through the provision of better methods 
for estimating the consequences of the various alternatives, 


In the initial stages of the application of statistical 
quality control methods in any organization, many changes in 
existing procedures will be proposed, The advantages of a 
proposed change in procedure will often be so great that the 
economic merits of the proposal will be clear without any for- 
mal evaluation in an economy study, In the following discuse 
sion, Cases A and B are intended to represent this familiar 
situation, Later on, after the more obvious opportunities 
for economies through statistical quality control have been 
exhausted, further applications may require careful economy 
studies; this is illustrated in Cases © and D, 


Case A, Routine Use of the Control Chart Provides an 
Improved Alternative, A certain measurable quality charace — 
teristic of a manufactured product is influenced by one or 
more adjustments of a machine used in the manufacturing proe 
cess, From time to time one article just off the machine is 
measured by the operator or by an inspector, If the measured 
value of the quality characteristic of this one article is 
either above or below the aimed-at value, the machine is ad- 
justed accordingly. 


This case describes a condition that often exists where 
manufacturing is conducted without benefit of statistical 
quality control, For example, the quality characteristic in 
question might be the weight of the contents of a food cone 
tainer, the shear strength of a spotweld, the inside perimeter 
of a paper bag, or the powder charge in a round of ammunition, 
Anyone familiar with the techniques of statistical quality 
control recognizes that the frequent machine adjustments made 
by the operator will inevitably increase the variability of 
the process, The routine use of the control chart provides 
an improved method of determining when machine adjustments 
shall be made; the use of X and R charts can be coupled with 
instructions that the process is to be left alone unless a 
point falls outside of control limits, The monetary advantages 
of the resulting quality improvements are sometimes spectacular, 


Case A is representative of the frequent circumstance 
in which the improved alternative to existing practice is 
provided by the routine systematic use of one of the tools of 
statistical quality control, 
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Case B, Statistical Analysis Makes Possible the Evale 
uation of the Consequences of Different Alternatives. A cer- 
tain quality characteristic of a manufactured article is exe 
amined by a destructive test that cannot be completed until 
several days after the production of the article, A substan- 
tial number of articles are tested from each day's production, 
The exact order of production is preserved until the results 
of the tests are known, Whenever an article tested fails to 
meet the quality specification, all of the articles are scrapped 
that were produced between the last preceding test specimen 
that met the quality specification and the next succeeding 
test specimen that met the specification, This acceptance 
procedure results in the scrapping of more than five per cent 
of the product. It also involves substantial testing costs, 
including the costs of the items destroyed by the testing proe 
cedure itself, 


The introduction of the techniques of statistical quality 
control into this manufacturing organization is accompanied by 
a statistical analysis of a large volume of past test results, 
This analysis, using control charts and other statistical tests, 
indicates that the test data are consistent with the hypothesis 
that a constant system of chance causes has been operating 
during most of the period of record, In other words, most of 
the articles scrapped did not differ materially from the arti- 
cles accepted, The articles accepted have, generally speaking, 
proved satisfactory for the purpose intended, This analysis 
indicates that there should be a thorough review of the pro- 
duction methods, the quality specification, and the acceptance 
procedure, 


Meanwhile a decision must be made regarding the accepte 
ance procedure to be used in the immediate future, One sug- 
gestion is to continue the past program of testing until this 
thorough review of the matter is completed but to make an im~ 
mediate change in the acceptance criteria, It is suggested 
that all of the product, rather than only part of it, should 
be accepted unless there is some specified change in the aver- 
age test value or in the variability of the test results, 

Not much of an economy study is needed to show that this suge 
gestion is superior to continuing the present acceptance 
criteria, As the statistical analysis of the past results 
has indicated that the rejected product was as good as the 
accepted product, and as the accepted product has generally 
been satisfactory, it is clearly more economical to accept 
all the product than to continue rejections on the present 
basis, 


This immediate decision in Case B is representative of 
the frequent circumstance in which the consequences of two 
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proposed alternatives affecting quality cannot be evaluated 
without statistical analysis. 


Case C, Are Extra Production Costs Justified by the 
Prospect of Fewer Lot Rejections? This case might be des- 
cribed as the case of the statistically sophisticated vendor 
. versus the statistically unsophisticated purchaser, A certain 
quality characteristic of a manufactured article is tested by 
a destructive test. The purchaser‘s specification states that 
the minimum acceptable value of this characteristic is 1,200 
units. (The characteristic in question may be thought of as 
strength, or as any other characteristic measurable only by a 
destructive test.) Actual acceptance inspection is on a lote 
by-lot basis with acceptance or rejection of a lot dependent 
only on the evidence of the sample from the particular lot, 
A single article chosen at random from each lot is tested, 
If the test value is 1,200 or more, the lot is accepted, If 
not, a second article is tested, If the second test value 
also falls below 1,200, the lot is rejected; otherwise it is 
accepted, 


The manufacturer of this product introduces statistical 
quality control techniques into his plant. A control chart 
is applied to this particular quality characteristic, With 
the routine use of the chart, it turns out to be fairly easy 
to bring this process into a state of statistical control, 
However, with present production methods the average value is 
about 1,375 and the standard deviation is about 100, If it 
is assumed that the distribution is roughly normal, this means 
that about four per cent of the statistically controlled prow 
duct will fall below the specified minimum value of 1,200, 


This matter is reviewed by the production engi.-¢ers of 
the manufacturing company to determine what changes are nec- 

essary for all of the product to meet the 1,200 specification, 
Two alternative measures for improvement are considered, 
Alternative I is to use a more costly raw material; it is 
believed that this will raise the average to at least 1,500 
with no change in the standard deviation, Alternative II is 
to purchase certain new equipment; it is believed that this 

: will cut the process dispersion in half so that the standard 
deviation will be 50 instead of 100, The extra cost of the 
more expensive material in Alternative I will increase proe 
duction costs by seven per cent, The investment charges on 
the new machine in Alternative II will increase production 
costs by nine per cent, 


A review of the costs of these two alternatives leads 
to a suggestion of Alternative III. This is to make no change 
in the production process and to continue to submit product 
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that is, on the average, four per cent defective, It is evi~ 
dent that if this alternative is selected there is only one 
chance in 25 that a single test specimen will fall below the 
specification limit of 1,200, With the acceptance procedure 
in force under which two speciments from a lot must both fall 
below 1,200 for the lot to be rejected, there is only one 
chance in 625 that any submitted lot will be rejected, This 
appears to involve a negligible cost compared to the cost of 
improving the product quality by either Alternative I or 
Alternative II, Hence Alternative III is clearly the most 
economical of the three, 


study in Case C resulted in a deliberate decision to continue 
to produce and to submit for acceptance a statistically cone 
trolled product with four per cent of the product below the 
specification limit, This decision was based, in part, on an 
evaluation of the probable frequency of rejections under a 
weak acceptance procedure, I have personally heard of only 
one case where a statistical analysis of a production process 
and of the acceptance procedure resulted in the decision that 
it was economical to take advantage of the weakness of the 
acceptance procedure, With the spread of statistical quality 
control, however, it is inevitable that a great many producers 
will be faced by this type of decision, Even with the best 
of sampling acceptance procedures, it is appropriate for a 
vendor to give consideration to the operating characteristic 
curve of the sampling acceptance plan and to weigh the risks 
of lot rejection, Weak sampling acceptance procedures provide 
a strong economic incentive to submit product with a moderate 
percentage defective provided it is possible to maintain 
statistical control of the production process and provided 
the cost of reducing the percentage of defectives is high in 
relation to the probable costs associated with rejected lots, 


Under What Circumstances Can Satisfactory Results Be 
Obtained With Weak Acceptance Procedures? The weakness of 
the acceptance procedure described in Case C is pointed out 
in nearly all introductory presentations of the basic cone 
cepts of statistical quality control, For instance, it is 
evident that if many lots 50% defective are submitted for 
acceptance, three-fourths of them will be accepted, Nevere 
theless, there has been widespread use of this acceptance 
procedure as well as many others that are somewhat better 
but are similar in giving little protection against the ace 
ceptance of lots containing a substantial percentage of pro- 
duct that does not conform to specifications, 


In spite of the limitations of such acceptance schemes 
when subjected to statistical analysis, we are doubtless all 
familiar with circumstances where they are used with satise 
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Does Case C Refer to a Common Situation? The economy 


factory results, That is, the results are satisfactory in 
the sense that there is no great complaint about the quality 
of the articles accepted, It is helpful to consider the dif-~ 
ferent possible circumstances under which such satisfactory 
results may be obtained, They are as follows: 


Either: (1) All of the articles in the submitted lots 
conform to the quality specification, Here the only effect 
of the acceptance procedure is to exert pressure on the mane 
ufacturer to continue this desirable condition, As implied 
by Case C, this pressure is strongest if the manufacturer is 
relatively unsophisticated as far as statistical knowledge 
is concerned and is therefore unaware of the weakness of the 
acceptance scheme, 


Or: (2) -The manufacturing process is badly out of stae 
tistical control so that submitted lots are either relatively 
very good or relatively very bad in their conformity to the 
quality specification, 


Or: (3) The quality specification contains a substane 
tial margin of safety. The lots accepted contain varying 
proportions of non-conforming articles. However, because of 
this margin of safety, most of these non-conforming articles, 
although technically defective according to the specifications, 
are good enough for the use to which they are put, Where an 
acceptance procedure is weak, the weakness may be offset, at 
least in part, by a design specification that requires articles 
that are better than actually necessary for the purpose at hand, 


Often the satisfactory results obtained at present with 
weak acceptance procedures are a result of (2) lack of statise 
tical control in the product submitted for acceptance, in com- 
bination with (3) a large margin of safety in the design spece 
ification, As more and more industrial products subject to 
sampling acceptance inspection are produced under conditions 
of statistical control, it will become less and less possible 
to rely on the lack of statistical quality control of submit. 
ted product to provide satisfactory results from weak accept- 
ance procedures, Hence improved control of production proce 
esses will require improved acceptance procedures. This change 
will also tend to bring under review a matter that has not yet 
been given much consideration in many industries, the matter 
of the economic relationship between design and acceptance 
specifications, 


The Relationship Between Design and Acceptance Specie 
fications, Shewhart, in his early writings, was doubtless 
the first to emphasize the distinction between a design 
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specification and an acceptance specification, The design 
specification, referred to earlier in this paper as the qual- 

ity specification, involves a statement of what is desired in 

the various quality characteristics of a product. The accept- 
ance specification deals with the means of judging whether 

the desired qualities are actually obtained, These two spec- 

ifications should properly be viewed as interrelated matters, 


The greater the margin of safety above what is really 
needed that the designer puts into his specifications of qual- 
ity characteristics measurable as variables, the less severe 
the acceptance procedures need to be in order to secure prode 
uct that is actually satisfactory for the purpose intended, 
This statement is not restricted to those circumstances in 
which weak acceptance procedures are used but is also applic- 
able when the sampling acceptance procedures in use have been 
developed with the benefit of the statistical approach, In 
modern acceptance sampling, the starting point for the decie 
sion on the acceptance specification is often the determinae 
tion of a percentage defective to be used as an acceptable 
quality level or as an average outgoing quality limit, One 
of the elements in deciding on an AQL or an AOQL is the margin 
of safety written into the design specification, 


Three General Classes of Costs Entering into Economy 
Studies Involving Quality Matters, In determining design 
and acceptance specifications, or in making any other decision 
relative to quality matters, it is advantageous to think of 
three general classes of costs that may be influenced by qual- 
ity decisions, These may be somewhat loosely referred to as 
(1) production costs, (2) acceptance costs, and (3) unsatis- 
factory-product costs, 


In this usage, the expression production costs is in- 
tended to refer to those costs involved in the production of 
the article to which the design specifications and the accepte 
ance specifications apply, Different design specifications 
may require different materials, different labor skills, dife 
ferent amounts of labor time, and different machines, For 
example, increased strength requirements for a part may change 
the material to be used; closer required tolerances on dimen-. 
sions may call for the use of newer or different machines, 
This general class of costs properly includes the production 
expenses on all product that is discarded as not meeting spec- 
ifications, It also includes sorting costs, if any, on re- 
jected lots, and costs of rework necessary to make product 
acceptable, 


The acceptance costs include not only direct testing 
and inspection costs but also the costs of administering the 
acceptance program, 
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The expression unsatisfactory-product costs is intended 
to refer to those costs resulting from the acceptance of product 
that turns out to be unsatisfactory for the purpose intended, 

It should be recognized that product that is technically dee 
fective in the sense of failing to meet design specifications 
does not necessarily lead to unsatisfactory-product costs if 
there is a sufficient margin of safety in the design specifi- 
cations, Of these three classes of costs affected by quality 
decisions, this class is inherently the most difficult to 
evaluate, 


Doubtless the greatest difficulty in evaluating unsatis-~ 
factory-product costs occurs in the consumers’ goods industries 
where the product goes to a great many different customers who 

do not apply any formal acceptance tests to the product that 

they purchase, It is hard to predict the consequences to the 
manufacturer of consumers’ goods when some stated percentage 

of his product fails to give satisfactory service to its pure 
chasers, and it is even more difficult to place a money value 

on these consequences, Where the consumers’ product carries 

a guarantee, past customer service costs can be used as a guide 
to judgment, and changes in these costs can be carefully watched 
and related to changes in design specifications and in inspection 
and acceptance procedures, 


The most favorable circumstances for arriving at a judg- 
ment on unsatisfactory-product costs exist where all of the 

product goes to a single user who has prepared the design and 
acceptance specifications, 


Case D. An Economy Study to Determine Design and Accept- 
ance Specifications. A manufacturer desires to make an economy 
study to establish design and acceptance specifications for a 
particular manufactured part where strength is an important 
Quality characteristic, This strength must be determined by 
a destructive test. As this product is made in batches with 
the possibility of considerable variation from batch to batch, 

. acceptance or rejection applies to entire batches, The objecte 
ive of the design and acceptance specifications is that each 
part shall have a strength of at least 2,000 units, and that 

: this result shall be achieved with a minimum total cost, all 

things considered, 


A number of different possible combinations of design 
and acceptance specifications are proposed, For purposes of 
discussion, let us represent these various proposals by two 
extreme alternative suggestions, Alternative I involves a 
margin of safety of 1,000 units in the design specification, 
so that a minimum strength of 3,000 units is required. The 


1 


associated acceptance specification is simple. One article 
is tested from each batch; if the strength is 3,000 or more 
the batch is accepted; if less, the batch is scrapped, 


The proposed acceptance specification in Alternative II 
involves a control chart for average and range and treats the 
test results as variables rather than as attributes, giving 
consideration to dispersion as well as to average value, It 
makes use of the modern ideas of varying the severity of the 
acceptance criteria and the amount of the testing with the 
quality history. Because of the additional protection given 
by better acceptance procedures, the margin of safety written 
into the design specification is reduced from 1,000 to 250, 


An economy study to compare Alternatives I and II requires 
consideration of the effect of each on production costs, on 
acceptance costs, and on unsatisfactory-product costs, 


If less costly production methods are required because 
of the smaller margin of safety in the design specification in 
Alternative II, this cost saving for Alternative II should be 
evaluated, The cost of product scrapped because of rejection 
of batches will need to be estimated for each alternative, 
This estimate must start with consideration of the expected 
distribution of the strength values with each quality objective 
(i.e., with the 3,000 minimum of Alternative I and the 2,250 
minimum of Alternative II) and an estimate of the extent to 
which the process will fail to show statistical control, It 
is also necessary to take account of the operating character- 
istic of each acceptance plan, 


In considering the acceptance costs influenced by the 
choice between the alternatives, the total amount of testing 
-under each plan will need to be computed, This can be deter- 
mined exactly for Alternative I where the amount of testing 
is fixed as one article per batch; the prediction for Alter- 
native II requires an estimate of the proportion of the time 
the process will qualify for reduced inspection, It is also 
necessary to recognize the greater administrative costs assoc- 
lated with the more complicated acceptance scheme of Alter- 
mative II, 


The evaluation of the unsatisfactoryeproduct costs must 
start with an estimate of the percentage of unsatisfactory 
product that will be accepted under each alternative, In this 
estimate, the proportion of accepted product falling below 
the design specification limits (3,000 and 2,250 respectively) 
is of no importance; it is necessary to predict the proportion 
of product falling below the actual requirement of 2,000 that 
will be submitted and accepted under each alternative, It is 
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also necessary to place a money valuation on the costs assoc- 
iated with the acceptance of a unit of unsatisfactory product. 


The next step in the economy study is the summation of 
the three classes of costs for each alternative -=- not only 
for Alternatives I and II but also for any other combinations 
of design and acceptance specifications that are under consid- 
eration, A comparison of the cost sums may eliminate some 
of the alternatives but may indicate that other alternatives 
deserve further study before a fimal decision. As these com- 
puted costs depend in part on a particular assumption regard- 
ing the centering, dispersion, and stability of the production 
process, it may be necessary to consider how the costs would 
behave with several different assumptions about the process, 


The final step in the economy study is a choice among 
the alternatives, giving consideration both to the monetary 
comparison and to any other relevant differences between the 
expected consequences of the alternatives that were not ex- 
pressed in money terms, 


Some General Comments on Economy Studies Relating to 
Design and Acceptance Specifications. Cases A and B described 
situations familiar to most of you; if parallel conditions 
have not existed in your own plant, you have known of them in 
some other plant. The general conditions that might give rise 
to a Case C are also familiar, even though you may not have 
knowledge of an economy study that reached the stated conclusion. 


In contrast to this, I know of no one who has actually 
made an economy study such as that described in Case D, This 
case is intended to suggest a logical development from the 
increased use of statistical quality control in industry, 

You may therefore quite properly view Case D with a critical 
eye. 


Several objections might be made to the suggestion that 
design and acceptance specifications should be viewed as ree 
lated parts of a single problem of economy and that in many 
Cases economy studies should be made to establish these spec- 
ifications, 


One possible objection is that this requires the designer 
to get together with the adminstrator of acceptance procedures; 
when they do get together it is clear that both must approach 
the matter from the statistical point of view, Although the 
people who administer acceptance procedures are rapidly acquire 
ing this viewpoint if they do not have it already, this is not 
necessarily true of designers, In those places where design 
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engineers continue in a naive and unrealistic view of statise 
tical matters, it will doubtless be impossible to secure sate 
isfactory consideration of design and acceptance specifications 
as related parts of a single problem, 


A second possible objection is that the consequences of 
various alternative proposals for design and acceptance spece# 
ifications are difficult to forecast and to express in money 
terms, This follows the pattern of a common objection to the 
making of economy studies as a guide to decisions in industry. 
The answer to this objection is that forecasts of consequences 
of decisions are the stuff of which decisions should be made, 
There is an implied forecast in every "hunch" decision, just 
as there is a conscious forecast in an economy study, 


A third objection is that economy studies such as the 
one suggested in Case D may be complicated and therefore 
costly, This objection raises an important point; any sorts 
of studies leading to decisions should pay their own way in 
the sense that the saving from the resulting improvement in 
decisions should exceed the cost of the studies. It seems 
likely, however, that industry affords many opportunities 
for such savings through improvement in decisions on quality 
matters, 


The Place of the Economy Study in Dealing with Quality 
Problems, This paper has not attempted to outline all of the 
ways in which the viewpoint and techniques of statistical 
quality control can contribute to economy studies involving 
quality. It has rather attempted to present certain provoc- 
ative ideas through the medium of a discussion of a few gene 
eral types of cases. A general conclusion is that, as time 
goes on, more and more quality problems in industry are likely 
to be approached by the techniques of engineering economy, 

The people who undertake such economy studies are likely to 
have an interesting time as well as contributing to the worthy 
objective of better industrial product at lower cost, 
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SPECIFICATION, MEASUREMENT and CONTROL of QUALITY 
by 
A. G. DALTON Fellow, A.S.Q.C. 


Superintendent, Quality Control 
WESTERN ELECTRIC COMPANY. 


The continuing success of any manufacturer or supplier 
depends upon how well he keeps his customers satisfied. This 
means that manufacturers and suppliers must know what will 
keep their customers satisfied. For some products, high 
quality is paramount, and in others, low cost is the predominant 
desire. However, in the vast majority of instances, both 
quality and cost are important factors in the customer's 
mind, and both must be controlled at satisfactory levels. 

In order to control anything, there must be a clean cut 
objective or specification, a means of measuring how well the 
obtective is being met, and there must be positive action taken 
when the objective is not met. Everybody knows what "cost" is - 
and how to specify it, how to measure it, and how to control it, 
However, when it comes to specifying, measuring, and controlling 
quality, we are up against a much more intangible subject. 
QUALITY CHARACTERISTICS 

Let us consider this word "quality" for a minute. 
There are two basic and important concepts of quality - one of 
these applies to a reality entirely independent of relation- 
ship to anything else or to our needs and wants. The other, 
and most common concept of quality, has to do with how we 
react as a result of contact with this real quality. The 
latter concept of quality is measurable in terms of the physical 
properties of things that make them desirable or useful to us. 


For example - weight, length, speed of operation, etc. It is 


lin 


necessary that we recognize the difference between these two con- 


cepts and concentrate on the latter if we are to satisfy the 
people who will use our products. 

How are we best satisfied? We might say that our 
opinion of satisfactoriness depends on what we get compared with 
what we expect to get for the price paid. We look for general 
satisfactoriness, of course, in a broad sense, but the fact is 
that adequacy, dependability and cost also tend to influence 
our overall opinion of the satisfactoriness of things. Naturally 
the relative importance of these factors will vary quite widely, 
depending upon the nature of the product; and our engineers, 
in writing specifications, must have in mind the relative emphasis 
that should be placed on adequacy, dependability and cost. 

In many instances, a high degree of uniformity and 
interchangeability is of major importance. Even in those cases 
where the need for interchangeability is not obvious, we still 
find that we are creatures of habit and react favorably to con- 
sistency. We prefer that the things with which we come in contact 
and with which we have been satisfied in the past, look the same, 
feel the same, or behave the same each time we come in contact 
with them. If they differ, to a recognizabledegree, from 
kindred products with which we have been satisfied in the past, 
we are inclined to be skeptical of their quality. 

Unfortunately, it is impossible to make two or more 
things exactly alike, however hard we try. When we are striving 
for customer satisfaction and approval, even in things which might 
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vary one from another quite a bit without much danger of these 


differences being detected, we try to keep them as much alike as 
practicable. When we are dealing with precise measurements and 


characteristics, and when a high degree of interchangeability is 
needed, we must call on all available resources to keep the varia- 


tions to the required minimum. 


Apart from the customer's needs, 


too much variability between parts - particularly in complex 
products - can cause many een 4 and unnecessary expense in 
subsequent assembly operations. | 

Recognizing the impossibility of making two or more 
parts exactly alike, design engineers provide tolerances for 


permissible variations. 


These margins may be set either to 
facilitate a subsequent mating of parts in assembly, or to 
provide some desired quality objective in the finished product. 
It is essential in this connection, of course, that the establish- 
ment of such tolerances include consideration of the capabilities 
of the tools and facilities which will be used for production. 


However, requirements of this nature, covering tolerances for 


individual parts, are not sufficient in themselves. A complete 


specification of quality objectives should include requirements for 
the aggregate quality of a multitude of such products, considered 


. as a whole. Such requirements which are often considered es 


inspection requirements, as contrasted with design requirements, 
can be specified in a number of ways - the most common being in 
terms of permissible percent defective - although there are many 
instances where they can best be specified in terms of the average 


and distribution. 
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It is important in this commection that engineers 
realize that quantitative requirements which cannot be verified, 


cannot be relied upon to control quality. For example, if 
minimum point thickness is desired for a protective coating, 

the design specification must also indicate the conditions of 
test under which this requirement may be considered to have 
been met. Supplementary information of this kind is also 

necessary when requirements cannot be verified because the 
specified limits for a characteristic represent destruction - 


as, for example, in the case of a requirement that a fuse must 


operate under a load of three amperes. Proper specification of 
requirements of this nature necessitate a knowledge of some of 
the basic principles upon which quality control techniques are 
based. 


There is one further point that should be stressed in 


connection with specifications and that is the impracticability 
of making them entirely self-sufficient. A specification is 
one of the means by which a Design Engineer attempts to convey 


the essential elements of the design intent to the people who 


are going to make and inspect the product. It may even be looked 
upon as a contract, but we know that a contract is not legally 
binding unless there is a meeting of the minds of the contracting 
parties. In order to obtain the most satisfactory results, the 

man who writes the specification and the man who makes the product 


must each know what is expected of him. 
the same language. 


the things that will be done during manufacture, even though 


They must both speak 
There should be a mutual understanding of 


they may not be included in the specification. For example: 


ie 


the precautions that will be taken to assure good workmansnip, 
good housekeeping and freedom from dirt, dust, solder splashes, 
ete., where such things could adversely affect the quality of 
the product. 
There should also be a mutual understanding of what 


is meant when tolerances for parts are included in the specifica- 
tion. For example: a specification for a shaft might tusinde 
the following requirements: 
Length - 4.000 + .005". 
Diameter - 0.500 + .0O01". 
Does this mean that a single shaft can be considered 
as meeting the specification requirements if it is 4.005" long 
but tapers a little bit so that it measures .501" in diameter 
at one end and .499" in diameter at the other end? Does it 
mean that the shaft need not be round, and that two measurements 
of the diameter taken 90 degrees apart may vary as much as .002"? 
Or does it mean that all shafts should be round, not oval and 
that the diameter of each shaft should be practically uniform 
throughout its length? Does it mean that in any lot of shafts 
it is all right to have some that are .499" in diameter and others 
-501" in diameter? The Design Engineer may have one idea and 

the Manufacturer may have another. This is just an illustration 


of the need for a common understanding among design, production 


and inspection personnel and the obvious economic advantages 
of being sure that this understanding exists before any large 
amount of product is manufactured. 


MBASUREMENT 


Let us now go to the second item - the measurement of 
quality. When it comes to the measurement of quality, we think 
in terms of inspection and inspection results. 

In thinking of inspection we must also think about 
inspection costs and inspection ratios. However, before we can 
begin to make intelligent comparisons of inspection costs and 
inspection ratios among different production units, we must know 
pretty well what part of the job is classified as "inspection" ana 
what part is classified as "operating". It is quite likely that 
in certain Shops work, that in other places is regarded as "inspec- 
tion", is classified as "operating" and the reverse condition may 
also exist. Furthermore, if, following comparison of inspection 
ratios, management pressure is exerted to reduce these ratios, 
there may be a tendency to reclassify operations formerly con- 
sidered as "inspection", as "operating", with an indicated 
improvement in inspection ratio. This may tend to relieve 
pressure where it is needed and it may exert pressure where it is 
not needed. 


The idealistic view would be to consider all work 


necessary to make product of satisfactory quality as "operating" 


and to consider "inspection" as that work necessary to verify 

the fact that the quality of product is satisfactory. Reduced 

to its simplest terms this would mean that if a part, sub-assembly 
or product was not of good enough quality to sample, then some 
part of the work which should be classified as "operating" was not 
adequately performed. 
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Let us take a simple example such as soldering wires to 
terminals. Our ultimate objective, of course, is to get satisfactory 
soldered connections. We can set up the job in either one of two ways: 

(a) We can instruct the operator who actually solders the 
connections to look at each connection immediately 

after it has been soldered to see that it is satis- 

factory before proceeding to solder the next connection. 

If it is not satisfactory the operator will, of course, 

re-solder it and make it satisfactory, or 

(b) We can have one operator do nothing but soldering and 

then have another operator look at each connection to 

see that it is satisfactory and arrange for the correc- 

tion of any which are found to be unsatisfactory. 
Whichever way we do the job, up to this point, we have done just 
one thing and that is produced a lot of soldered connections which 
we assume are of satisfactory quality. The next step is to verify 
that this assumption is correct and this verification work should 


properly be classed as "inspection", since it is nothing but a 


measuring job and adds nothing to the quality of the work that has 


been performed. 

When performed as a means of verification, inspection 
usually involves an examination of representative samples of 
product. In such cases two risks will be involved, due to the 
chance that the selected sample may not be sufficiently representa- 
tive. In other words it may not contain the same percentage of 
defective units as the lot from which it was drawn. These risks 
may be considered as a consumer's risk and a producer's risk. 

The first represents the risk of accepting product of marginal 
quality and the latter the risk of rejecting product of aimed-at 


quality. 
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When sampling is used for quality protection, the two 


plans most commonly employed are "lot quality protection" and "average 
quality protection". Each involves rejection or acceptance of a 

lot of product depending upon whether the number of observed 

defects is greater or no greater than the "allowable" number for 

the particular sample selected. 

Under proper conditions, sampling inspectiom is not 
only cheaper than detailed inspection but is usually more 
accurate and more informative. Determination as to whether or 
not conditions are appropriate for sampling and, if so, which 
type of sampling plan should be used, must include consideration 
of the following factors: 


1. Are we interested in limiting the acceptance of more 
or less isolated lots of product to those lots which 
contain no more than a specified maximum percent 
defective? Or are we more interested in controlling 
the average percent defect of a series of lots of 
product which may later lose their identity in 
subsequent operations, in stockrooms, or in shipment? 


2. What relation exists between the actual percent defective 
that results from our production process and the values 
we wish to use, either as limiting values for individual 
lot protection or for average outgoing quality protection? 
If the process average percent defective exceeds the 
limiting value within which we wish to control the 
average outgoing quality, or exceeds one half of the 
value established as the maximum for any lot, sampling 
is usually not economical. As the process average 
percent defective approaches these levels, we can no 
longer enjoy the benefits of small sample sizes and 
still retain a low producer's risk. This means that 
we are forced to spend more money, either in taking 
larger samples, or in fixing up the greater number of 
lots that will be rejected if we hold to a policy of 
continuing to take small samples under these conditions. 


One of the major factors in keeping our process average 
percent defective at a low figure is, of course, the elimination 
of those causes of variability which are not due to chance alone. 


9. 
Such items as abnormal variations in materials or machines, lack 
of skill, or carelessness in manual operations, rough handling, 


etc. are typical other-than-chance causes. Proper analysis of 


our inspecticn results will disclose the presence and magnitude 


of the effect of these other-than-chance causes. The more rapidly 
they can be eliminated, the more rapidly can we go back to smail 
samples and enjoy the resulting economies of this form of inspec- 
tion. By the same token, improvement in the process average 
percent defective, by the elimination of these other-than-chance 
causes, will result in less re-work, less scrap, and a correspond- 
ingly smaller unit production cost. It should be kept in mind, 
however, that there may be circumstances where even after the 
other-than-chance causes have been eliminated, the process average 
percent defective is still too high. This means that we have 
control, but the control is around the wrong level. The only 
thing to do under these circumstances is to change our facilities 
and processes, or to change our objective quality level. 
RELIABILITY 

Before we leave the subject of inspection, I would like 
to emphasize one more point, and that is reliability. Since 
successful application of quality control principles depends 
very largely on inspection and inspection results, our primary 
concern should be that the inspection results are as reliable as 
possible - whether they are used for protection or for informa- 
tional purposes. 

Here we are faced with a very serious problem. The only 


way we can secure entirely accurate information with respect to any 


given number of parts or things, is to inspect each of them under 
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conditions where there will be no error in measurements or 


judgment. The opportunities for operating inspections under such 


conditions are, of course, extremely limited. In practice we 


have indeterminate inaccuracies in the results of almost any 100 


percent inspection, due to human error. On the other hand, the 
P results of sample inspections may not always provide a true indica- 


tion of quality owing to the laws of chance. Fortunately, however, 


in sampling inspections we can calculate the influence of chance, 
and the effect of the human error is likely to be considerably 
less because of the interest incentive associated with sampling 


inspections. 


Another, and very important, factor influencing the 


reliability of inspection results, is the point of view of the 
inspector. Werever possible, inspectors should be insulated from 
exposure to the day-by-day problem of production schedules and 

costs. They should try to put themselves, mentally, in the 
position of the recipient of the product - whether it be the 


customer or the production group which is going to use the product 


as a part or sub-assembly in the next operation. If any inspector 


allows himself to be influenced ever so slightly by a desire to 
contribute to the output or cost bogies of a production group, 


because he is one of the gang, he is likely to look less critically 


at marginal cases, and if at the end of the week or month output 
and cost objectives appear to be in jeopardy, he may also assume 

that things slightly outside of limits are also marginal, and that 
the shop or production unit should be given the benefit of doubt. 
In sampling inspections where there is an allowable number of 


defects for a given sample size, these emotions and influences may 
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result in an inspector regarding one over the permissible 


number of defects as the workings of an unkind fate which threw 


the extra defect into the particular sample he selected. This 
can very easily lead to the conviction that had he taken a 


different sample he would not have found one over the allow- 


able number of defects and it should, therefore, be disregarded. 


This dissertation on the reliability of inspection 


results is intended to highlight the effect of the point of 


view and the influences of emotion on the inspector rather than 


any question of honesty or dishonesty. It is important to 


recognize that unconscious inaccuracies can, and probably will, 


be present in inspection results if the inspector, or his 


immediate supervisor, has to meet cost or production schedules. 


Under such circumstances, it is too much to expect that 


inspectors can consistently retain the necessary unbiased point of view. 


While I have included such expressions as "protection" 
and "control" under this discussion of inspection, I wish to 


make it crystal clear that inspection per se, is nothing but a 


measurement. It does not improve, protect or control quality. 


In fact too much inspection may well do the opposite. It is 


the action we taken on the basis of inspection results that 


improves or controls product quality. 


ONSID. 
What then do we get out of inspection? Is it worth 


what it costs? 
In trying to answer these questions it will simplify 


matters if we deal first with material, piece part, sub-assembly 
These may be divided into two 


and other process inspections. 


categories for discussion: 
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First, there are those cases in which a definite 


minimum process inspection should be performed regardless or 


how good current quality may be. This is in the nature of 
quality insurance and is especially necessary for those 


process requirements which are not, and possibly cannot be, 


verified by inspection of the finished product. Many of us 


drive a car year in and year out without being involved ina 
Single serious accident, yet we would not think of being without 
liability insurance because we know that if we ever were involved 
in one the results could be so seriously damaging that we refuse 


to run the risk. Likewise, if an important material or process 


got out of control without our knowing about it promptly, the 
adverse customer reaction that would probably result and the 
damage to ourselves in terms of reputation and replacement or 
repair costs would be so great that we should never go below 
a specified minimum inspection, how ever good current quality 
might be. 
The kind and extent of this minimum inspection should 
not, therefore, be determined by how many defects we have found 
or have not found on past inspections, but by an estimate of the 
damage which could result if the process did get out of control. 
The second class is much larger and includes those cases 


where something more than the basic minimum process inspection 


will prove to be worth while from an economic standpoint. Just 
how much more than this basic minimum inspection should be made 
depends upon actual guality, cost of the increased inspection and 


the value of the benefits gained. In general, manufacturing 


processes provide a certain amount of defective product which 
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results in scrap, repair and rework. The exact value of these 
losses will depend upon the amount of defective product and the 
particular stage at which it is found. Further losses will 

result from that portion of the defective product which inad- 
vertently may be furnished to the customer because the defect 


is not readily detectable in the completed state. It should not 
be too difficult to equate the cost of protective process 


inspection against the savings in scrap, repairs, rework and 


consumer adverse reaction. An optimum point at which the 


cost of process inspection is no greater than the cost of 
scrap, rework, etc., can thus be arrived at and the appropriate 


character and amount of inspection prescribed. 


It is only by 
applying careful consideration of this kind to the development 


and specification of inspection procedures that producers can 
be reasonably sure they are meeting their quality objectives 
as economically as possible. 
When it comes to inspection of finished products the 
dollar value return for each dollar spent on inspection is 


harder to identify and define. 


Against the cost of such inspection 


must be balanced the cost of complaint investigations, repairs 


or replacements which would result from selling unsatisfactory 
product,and also loss in reputation and good will. Some of 

these are rather difficult to evaluate but on definitely 
in the picture. 
Another advantage derived from reliable inspections 
is the probable improvement in producer - consumer relations, 
especially where consumers are willing to accept producer's 
inspection results as a partial or complete substitute for 


acceptance inspections that would otherwise have to be made 


at the consumer's expense. 
I should like to reiterate that in order to be 
reasonably sure we are getting our money's worth for inspection, 


the cost of each inspection operation should be weighed against 


the penalties of omitting the operation. These penalties can 


be considered as expense in terms of waste, abortive assembly, 


repair or replacement in our own plants, or adverse consumer 
reaction due to poor appearance, unsatisfactory operation, 
insufficient life, or any other failure to meet advertised 
claims. 
CONTROL 
Having specified our quality objectives and considered 
various methods of measuring quality, the next step comprises the 
eotablishnent of procedures for controlling product quality at 
the desired level. In planning a quality control program, the 
following questions warrant very careful consideration: 
At what points in the process from raw materials 


through sub-assemblies to completed prosust should 
controls be introduced? 


For how many individual characteristics should 
separate control charts be maintained? 


To answer questions of this kind requires sound 
engineering ability and experience, both with respect to the 
product characteristics and with respect to the production 
facilities and process. The program may be compared in its 


initial stages to preparing a questionnaire. Those of us 
who have at times attempted to get detailed information on the 
same subject from different sources, will realize that while 

the collection of the information may have taken a lot of | 
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people a lot of time and effort, when we tried to piece it 
together we have found that we had not gotten exactly what 
we wanted. The fault may well have been our own, particularly 
if we did not spend enough time deciding exactly what it was 
we wanted before asking a lot of people to get it for us. 

In setting up a quality control plan, we must then 
first decide exactly what kind of information will be most 
useful to us, in total and in detail. We can well afford to 
spend plenty of time on these considerations ~ ask ourselves, 
for example, how much will the information be worth when it is 
collected? What are we going to do with it? Which are the 
points in the manufacturing process which represent relatively 
large steps in the ladder of production costs from raw materials 
to the finished product? 

These considerations will involve intimate knowledge of 
the quality characteristics which are necessary for production 
operations - i.e., hardness, fit of parts, etc. - and also those 
characteristics which are necessary in the final product to 
satisfy the consumer. Unless these facts are very carefully 
considered, there probably will be a temptation to cover more 
characteristics and introduce more control points and control 
charts than are economically warranted. Especially in those 
instances where management has not had very much experience with 
quality control techniques, and might be alarmedat the preten- 
tiousness of a quality control program, it is well to start off 
with a small and modest plan. 

One way to develop a Keen management interest and 
appreciation of the walue of quality control, is, of course, to 
have these techniques pay dividends. Fortunately, in a manufactur- 


ing plant where these techniques have not been used and management 
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may, therefore, be a little skeptical as to their worth, there 
is likely to be a virgin field of opportwnity for quick and 


impressive savings. In such cases, it probably would be well 


to go after the cream and to aim quality control at some point 


where heavy scrap losses are occurring, or at a point which 


appears to be the focus of customer complaints. It becomes 
harder, of course, to continue to show new and important savings 
as we wrk our way through the cream and come down to the milk - 
but by that time we will have gained experience in how best to 
apply quality control techniques and can make more of these 
lesser opportunities. This is where the ingenuity of the 

quality control engineer is taxed most severely. It is, however, 
a part of quality control work which should attract high caliber 
engineers, and keep them interested. 
I should like to emphasize that even with adequate 


quality specifications and adequate means of measuring quality, 
wecannot control product quality without taking action, and 


sometimes very drastic action, when our measurements show 


definite evidences of unsatisfactory trends in quality. In 

other words, it is no use spending the money to build a satis- 
factory alarm system unless one heeds the alarm when it goes off. 
Those who are unwilling to slow up, or even stop production, if 
necessary, in order to determine and correct the cause, or causes, 
of unsatisfactory trends will find, in the long run, that the 
cheapest thing to do with unsatisfactory product, is not to make 
it. Here is where a sympathetic management is essential since a 
quality control program has little chance of success uniess it 
receives the wholehearted support of top management and all 
employees in the organization know that top management is 
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actively interested. 


In closing, I should like to summarize the items which, 


in my opinion, determine the degree of success of any quality 


control program. 


1. 


The completeness and definiteness with which quality 
objectives are specified - for individual units of 
product and also for "lots" of product. 


The cleverness with which control points are 
established and economic and efficient inspection 
plans are worked out for each control point. 

The reliability of inspection results. 


The ingenuity with which inspection results are 
analyzed and interpreted. 


The promptness and vigor with which corrective 
action is taken when inspection results show the 
need for action. 
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QUALITY CONTROL OF JET AIRCRAFT COMPONENTS 
Applications of Practical Statistical Quality Control Methods in the Shop 
By W. C. lee* 


This paper is devoted primarily to a description of practical shop 
methods for controlling quality during two types of manufacturing operations 
where conventional statistical procedures may not be easily applied. 

1) Slow production items from contimous operations. (From 1 to 
10 parts per hour) 


2) Intermittent, short-run, low-quantity batch-type manufacturing, 


Unlike high production items of average quality from continuous 
operations, where manufacturing conditions remain relatively constant, and 
a@ moderate quantity of completely rejectable parts can be tolerated, the 
production of small quantities of high quality aircraft material, at a low 
rate of production, under constantly changing conditions presents many 
problems and offers a real challenge to controlling quality by statistical 
methods. 

It is doubtful if there are many manufacturing plants in the country 
who do not have some jobs of this nature and these are the ones that some 
companies may have hesitated to place under §.Q.C. In controlling the 
quality of stainless steel components for Jet Aircraft Engines there is no 
choice, since they are all relatively difficult from a statistical stand- 


point. We believe, however, that the procedures we have adopted for these 
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jobs may be useful in other metal working industries having similar prob- 
lems. It is believed the illustrations to follow will prove that S.Q.C. 
need not be confined to high~production items alone and that it has a 
definite place in the low production field. 

Solar Aircraft Company supplies many of the heat-resistant components 
for these engines, including the Ring & Outer Tube Assembly which house the 
Combustion Chambers. (Illustration A) One of the first parts to be put 
under §.Q.C. at the Des Moines Plant is the Ringholder component. 
(Illustration B) It ig machined all over from stainless steel cast tubing 
(4MS 5645) at the rate of 6 per hour from each machine of a battery of 3 
Gisholt Turret Lathes, in continuous operation. Tool mortality is high 
and tool changes are frequent. Tool and machine cujustments are necessarily 
numerous. Before applying Statistical Quality Control, scrap and rework 
costs were necessarily high and inspection time was correspondingly excess- 
ive. This was not an ordinary subject as the variables were numerous and 
their frequency high. (Illustration C) 

There are several different methods for controlling quality ina 
manufacturing operation. One common method is to set up a Percent Defective 
Chart at an inspection bay and wait for the material to be inspected. Or 
we may inspect where the process is taking place. Sampling Inspsction can 
be used of course by some approved plan and the lots which exceed the per- 
missible percent defective may be screened and the defectives removed. 
After sufficient data has been accumulated so that the troublesome dimen- 


sions can be definitely identified, it is common practice to apply an 


Average and Range (X & R Chart) to the troublesome dimensions and attempt 
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to reduce or eliminate the assignable causes of defective material, main- 


taining the Percent Defective Chart to record the trend of the process and 


note any improvement. 

After the trouble has been found and presumably eliminated it is 
common practice to abandon the X & R Charts and revert to the Percent 
Defective Chart completely, after the trouble has disappeared or has been 
reduced to an acceptable minimun. 

We use this type of chart (Illustration D) wherever practical and it 
has been found very satisfactory, within its limitations. Note that the 
lower section of the sheet is a bonafide Percent Defective Chart and is 
plotted as such. The main columns list all of the characteristics to be 
inspected and the gages used for their inspection, comprising in effect, 
an inspection check list of the job. We record whether the material is 
undersize, oversize, reworkable, scrap or salvageable using the code shown 
on the chart. Since our inspection with a go and not-go gage tells us that 
it is undersize or oversize, we know whether it should be reworked, scrapp- 
ed or used on a salvage tolerance. This of course facilitates disposal of 
defectives. Quality Control personnel are furnished with a list of per- 
missible salvage tolerance to which we will accept off=print material 
(up to a given percentage.) Any material exceeding this salvage tolerance 
must be reworked or scrapped. 

Left of the gage column is the inspection plan. Since our plant is 
under Air Force surveillance inspection, we lean heavily to double sampling 
per JAN-105. The Acceptable Quality Level is specified, designating the 


percentage of defective material which may be accepted on any group of 
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similar characteristics having the same A.Q.L. 


Note that the A.Q.L. varies 


from dimension to dimension. A critical dimension would receive careful 


100% screening and the Acceptable Quality Level would be zero. Defects of 


@ major nature affecting the malfunction, useability or interchangeability 


of a part would have a low A.Q.L. usually 1% or 2%. Minor discrepancies 

caused by workmanship only, such as finish, sharp edges, etc. are accepted 
up to 5% defective, the material being considered useable. 
The purpose of this type of inspection procedure is to conform to 
Air Force requirements and to reduce stocking of any material which is not 


useable to a minimun. 


If the sample discloses any material beyond salvage 
tolerances the entire lot is 100% screened for that defect and the defect- 


ives removed for reoperation or scrapping. Unlike assembly methods by 


bolts and nuts, many of our components are welded to large assemblies and 


cannot be removed. We therefore cannot afford to stock unuseable material. 


The total number of parts reworked, salvaged or scrapped may be 
individually totalized and recorded at the base of each colum. The total 


number of defectives are entered in the appropriate block, the percent 


defective calculated and plotted as a regular P Chart. In totalizing the 


defects a part is considered only once in case two or more defects appear 


on the same part. For record purposes, the percent defective is estimated 


by dividing the number of defectives found in the first sample by the 


number of parts inspected in the first sample. 
Over a period of time it is possible, by adding from left to right, 
to totalize the number of defects for any one characteristic, and identify 


the more troublesome dimensions. By the same reasoning, if a high plotting 
3 


7 

‘ 


appears on the P Chart it is possible to move upward and immediately 
identify the troublesome characteristic, and at the same time to obtain 

the trend of that characteristic over a period of time. 

We have tried to incorporate in one form all of the information 
necessary for a preliminary study or a permanent record of any job. It 
is used in our Forming, Receiving Inspection and other manufacturing 
departments where inspection by attributes is applicable. Our Purchasing 
Department finds this type of chart to be of value in rating supplying 
vendors, in fact, vendors are supplied with a record of their performance 
by this method. Since a Percent Defective Chart records inspection by 
attributes, using fixed gages, it is an economical medium of recording 
material or process variation, particularly -where tolerances are liberal 
and tool changes or adjustments are not too frequent. 

The Percent Defective Chart has some limitations, however, when used 
on short-run or low-production jobe in a Machining Department where speci- 
fications are tight, tool mortality is high and the human element is a 
large factor, such as the following: 

1) The P Chart does not provide a sensitive guide for the operator 
except to warn him of the quality of the material being produced. (We have 
found that the psychological effect of the X & R Chart on the operator is 
one of the most important factors in §.Q.C.) 

2) It does not furnish a detailed record of tool wear, tool changes 
or machine adjustmente. This is valuable date as it has a distinct bear- 


ing on the establishment of inspection frequencies. 


3) It tekes a greater number of pieces to discover which dimensions 


are troublesome and by then the job is often well under way. While a P 


Chart will record the nature and direction of variations, it does not 


record their extent. On short run jobs it is important that this infor- 
mation be available as quickly as possible. 
4) It provides information as to the proportion defective—it 
does not provide information as to how good or how bad. 
So, while the Percent Defective Chart is a valuable economical tool 
for obtaining general information of a process, it has some limitations. 
It does tell us where to look for trouble and that of cource is the first 
step toward eliminating it. 


It does not furnish the accurate information 


obtained by a thorough etudy, using X & R Chart techniques. 


Undoubtedly there are many different ways of controlling quality. 


The method we prefer to use and which we chose to illustrate here, while 
perhaps not as theoretically or mathematically advanced as some others 
coming into use, it works in the shop and is paying off in reduction of 
scrap and reoperation costs. 
The procedure which we advocate for controlling quality on iow- 
preduction short-run items is based on abgolute and complete control of 
the procegsg. This sounds imposing and expensive, but actually it need not 
be. It is simply a procedure for finding, recording and analyzing the 

causes of process variation, reducing the excessive variations to an 
acceptable level, then maintaining control of those variations at all 
times. 
To be sure that this control is complete, it is first necessary to 


make a thorough study of the item to be placed under control. 
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do not mean merely to determine the percentage of defectives by 100% or 
sampling inspection, but to determine and record not only the nature and 
direction but the extent and reason for every variation of every dimension 
during the study. 

Without this type of study and control, we would never know the pro- 
cess capability of all dimensions in detail; how close to the borderline of 
rejection we were operating. 

Most important of all, we need sufficient evidence to apply the min- 
imum amount of inspection for a predetermined amount of protection, if the 
Quality Control program is to operate economically. 

Furthermore, lacking a complete study, we might lose the benefite of 
valuable by-products which are nearly always obtained through a thorough 
study of a job. 

After choosing the subject of our study, which is usually a high 
mortality item, the first step is to assign a competent Quality Control 
Engineer and a trained Process Inspector to the job. For several days 
every variation of every dimension is carefully recorded, particularly the 
reasons for abnormal variation. (Illustration B) All tool changes, tool 
adjustments and machine adjustments must be faithfully noted. Until we 
know how many parts a cutting tool will produce without a machine adjust- 


ment or a tool change we cannot accurately select our final sampling plan. 


No feed or speed changes should be permitted during the study period. 


(Note that space is provided for recording all tool changes, tool adjust- 
ments and major machine adjustments.) (Illustration F) 
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4s each new bar of stock ie put into use a recording is made on the 
chert. Set-up in the morning on cold machines is taken into consideration 
and the first few individual readings plotted as such. It is advisable to 
have 100% inspection for the first half-hour. About the third day sub-lot 
sizes and sample sizes can be determined and Sampling Inspection started 
on the less troublesome dimensions. As the process continues within the 
control limits established, Sampling Inspection can be gradually relaxed 
on the dimensions running well within control. However, more frequent 
inspections are maintained for the more troublesome dimensions. 

Corrective action ghould be taken on these troublesome character- 
istics until the cauges of excegsive variation are eliminated. Corrective 
action can take many forme. Improved methods or tooling; different speeds 
or feeds of the machine; operator training; machine maintenance; investi- 
gation for consistency of material; specification changes; realistic rather 
than idealistic inspection standards; coolant control; improvements in 
previous operations; all with the thought of improving, not decreasing 
manufacturing efficiency. 

When the study is completed and sufficient data accumulated, the 
control charts are removed and carefully analyzed by top Quality Control 
personnel. Where practical, dimensions having liberal tolerances and a 
known low range of machine variations are placed on a permanent Percent 


Defective Chart, maintaining an inspection after every tool change, tool 


adjustment or major machine adjustment. Average and Range (x & R)Charte 


can be held on a standby basis. They are retained on the most troubdle- 
some dimensions having larger process variations or tight specifications. 
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4 minimum process-inspection plan can then be established and it becomes 
permanent. 

Out of control plottings automatically require 100% screening of 
' that periods’ run and the largest and smallest individuals are considered 
for analysis purposes. Defectives are removed to a locked quarantine area. 
Reoperation of reworkable parts is performed before the set-up is torn 
down. Accurate records of reoperation are maintained. Prompt disposal of 
scrap is made to clear material records. 

Operators should be trained to call for inspection immediately 
following tool changes or machine adjustments, and process inspection 
should be made available for this inspection. It is cheaper to provide 
this process inspection than to 100% screen material produced during the 
regular inspection interval. With this type of control very few defectives 
will be produced. A very small sample, taken at frequent intervals offers 
the greatest protection and gives the operator assurance that the process 
is satisfactory. These small but frequent samples are far more valuable 


to the operator than large samples taken once or twice a day. Assignable 


causes of discrepant material cannot easily be found for work produced 


hours ago. 

Prior to applying S.Q.C. to this particular part a large percentage 
required reoperation for one reason or another, dimensionally or finish, 
before they could be presented for inspection. A thorough S.Q.C. study 
disclosed that all dimensional and finish requirements could be met with 
proper controls, and that final acceptance could be made at the machines. 
Reoperation has been reduced to a low figure. Final 100% inspection has 
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been completely eliminated, scrap losses are low and the parts go directly 
from the machines, through Zyglo Inapection into Stores. 

Illustration F (of one dimension) shows final controls establish- 
ed and in operation. Wine charts are presently maintained on this jobd, 
however only one part per hour is inspected and we expect to reduce the 
number of charts as the job improves. 


Since approximately 24 parts are produced every 4 hours, we could 


have chosen a lot size of 24 and a sample size of 4. We elected, for 


better control, to call our lot size 12 and our sample size 2, however one 
sample is inspected each hour and averaged each 2 hours for calculation 
purposes. This offers more assurance to the operator that the process is 
operating satisfactorily. 

Every S.Q.C. study usually results in some interesting by~products. 
This particular study resulted in the release of a South Bend Engine Lathe 
and a heavy duty Drill Press for other work, this equipment having been 
used for performing reoperation on these parts. Material used for re- 
finishing is no longer required. Handling has been reduced and special 
containers for these parts are more readily available. Large banks of 
parts are no longer neceesary as they are useable when they leave the 
machines. 

Manufacturing efficiency may suffer somewhat during a study and 
until final controls are established, however, in the final analysis a 
thorough study usually results in an increase in manufacturing efficiency. 

The same type of control works equally well for intermittent, short- 


run jobs as for low-production, continuous operations. On short-run jobs 
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however, it is well to know the production schedule intimately, have all 
control charts ready and Process Inspectors alerted. 

A study such as described reveals many conditions which contribute 
to excessive manufacturing time. Processing dimensions which receive sub- 
sequent machining are often held to unnecessarily tight tolerances and 
finish requirements. Dimensions which disappear in subsequent operations 
are often over-inspected. Illustration G shows the cross section of a 
part which was produced intermittently at the rate of two per hour. The 
same type of study was conducted as was described in the previous illust- 
ration. Among other things it was found that the operator was attempting 
to maintain a ~ .005 tolerance and a 63 finish on the flange faces and 
outside diameter, and was maxing an excessive number of tool changes and 
tool adjustments. The Average and Range (X & R) Charts (Illustration H) 
shows tool wear and tool re-setting in attempting to hold the close 
dimensions and fine finish. (Illustration J) 

The S.Q.C. study disclosed that these were processing dimensions 
which received subsequent machining and that the machining tolerance 
could be doubled to # .010 and the finish increased from 63 to 250 micro 
inches. (Illustration H) indicates that tool changes and adjustments 
have considerably decreased ag the result of these changes. 


Manufacturing efficiency has increased on this item from 77.9% 


to 102.5%, part of which resulted from the study and the balance from 


byproducts of the study. 
Some of the greatest savings can be realized from by-products of 
§.Q.C. studies, often more than by control chart applications themselves. 
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We stumble over them every day and do not realize they exist until reveal- 
ed as an afterthought to the original objective. 

Every time a tool is changed to improve the finish of a machined 
part it mist be handled several times. First it must be removed from the 
machine, delivered to tool grinding, ground, inspected, returned to tool 
stores, withdrawn by the operator and placed in the machine. Every time 
tool life can be increased it eliminates these steps of handling, regrind- 
ing time, and it reduces down-time of the machine producing the material. 

Set-up control is very important. A set-up man or operator not 
having run a job for a week or a month, may make a few scrap parts until 
a correct setting is reached, unless set-up control is made effective. It 
is common practice for set-up scrap to be retained under a bench until a 
lot is completed perhaps indefinitely, resulting in hidden loss and in- 
accuracies of material control records. With effective set-up control, 
set-up scrap can be reduced to a minimum but it requires the coordinated 
efforts of the set-up man, foreman and process inspector. The inspector 
must be there when the first part is made and the machine should aot run 
until it has been checked and the results known to the set-up man or the 
operator. 

Segregation of material from controlled processes is also very 
important. We have established accumlation areas in each department and 


as material is accepted by the process inspector it is placed in these 


areas. (Illustration K.) When a sufficient quantity has been accumulated 


a Process Inspection Release is attached to the material and it may then 


be moved to the next operation or into Stores. No one is permitted to 
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place material in an accumlation area except a process inspector and no 


one but a Production Control employee may remove material from these areas. 
Before each run is completed all defective material which has been 
held by the process inspector in a quarantine area, is reviewed by a 


Quality Control Engineer, the reworkable material is reworked before the 


set-up is torn down and the unuseable material is scrapped. This pre- 
vents odds and ends of defective material from accumulating and keeps 
stock records current. 
While a great deal of our work at the Deg Moines Plant consists of 


forming and machining of stainless steels nearly all of the completed 


units we produce are assembled by some form of welding. In the field of 


metallic-arc or acetylene-gas welding the greatest variable is the skill 


and consistency of the operator or the lack of it. In attempting to con- 


trol these variables we are actually controlling the individual rather 


than mechanical variables. 
Resistance welding on the other hand, contains many variables be- 
yond the control of the operator such as electric power input; air 
pressure, water vOlume, water temperature, electronic controls; condition 
of electrodes and the variation of inspectors’ opinions as to the accept- 


ability of the weld. 


Since the testing of resistance welding is necessarily destructive 


it is obviously impossible to 100% inspect the end product. Therefore the 
only method of assuring the quality of welded products is to control the 


: process. Visual inspection of resistance welding except for external 


defects is wishful thinking. 


Industrial X-ray is commercially uneconomical 
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for such inspection. Sonic testing for this purpose is yet to be developed. 

Inasmuch ag our welded assemblies require maximum structural 
strength, to Air Force specifications, we must be able to guarantee the 
quality of all welded products in our production lines. We have adopt- 
ed Statistical Quality Control methods on all resistance welding operations 
at the Des Moines Plant and the Resistance Welding Control Chart (Illust- 
ration L) shows the type of control which is maintained. Our Resistance 
Welding Inspectors are laboratory trained in all phases of physical test- 
ing and microscopic examination of welding, tensile testing, sectioning 
and micro-etching, mounting and polishing of specimens. 

In respect to training, the entire supervisory staff of our plant 
have received approximately 15 hours basic training in S.Q.C. fundamentals 
and applications. Hourly paid inspectors receive approximately 10 hours 
basic training in the use and plotting of control charts. 


The selection and control of Quality Control Personnel (Engineers 


and Statisticians) is important as their approach, manners and ability to 


make friends can materially aid a program. 

Our approach to the labor unions was very open and the negotiating 
committee and all union stewards attended a condensed two hour discussion 
on the fundamentals and purpose of S.Q.C. All of this was accomplished 
before any control charts were applied in the shop. 

We have found that it is wise to locate the first installation 
where gupervision is strong and receptive to progressive methods. 
Properly presented and diplomatically executed §.Q.C. will sell itself to 
intelligent supervision, for they realize that it will make their jobs 
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easier and their efforts more effective. Factory supervision realize that 
every process inspector is another pair of eyes for the foreman. It is 
essential to gain the confidence of production supervision and to give 
them full credit for their contribution to the cause of Quality Control. 


It is never wise to claim that the Quality Control Engineer did the job. 


It is very important to brief hourly paid operatore before apply- 


ing control charts to machines. Everyone is suspicious of something new 
that they do not understand. In this connection, machine operators were 
taken into a conference room and the principles and purposes of S.Q.C. 
were explained to them, and each worker was provided with a simple shop 
manual, supplementing the briefing. With this type of approach, griev- 
ances, because of S.Q.C. applications has been unknown at the Des Moines 
Plant. (Section of Manual shown in Illustration M). 

The assistance and guidance a control chart provides for a machine 
operator is an asset to the Quality Control Engineer which should not be 
underestimated. The psychological effect on the operator to produce 
acceptable material is one of the most valuable tools of S.Q.C. The fair- 
ness of the control chart appeals to the average operator, if we can make 
him feel that the chart and the process inspector are there to help him 
produce better work. 

To those who may be skeptical of the dependability of process con=- 
trol on slow=production or intermittent, short-run jobs, it may be said 
that the accuracy and dependability of this control depends to a great ex- 
tent upon the thoroughness of its preparation and the manner in which it 
is executed by the Quality Control group, the process inspectors and super- 
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It will be only as effective as it is prepared and 


vision in general. 


executed. We could cover the walls with control charts but if the oper- 

ators and the foremen, the men who produce this material, do not want them 
or use them effectively we have failed in our undertaking. 
For those who would attempt to reduce the necessary time to thoro- 
ughly study a job, we may say there are no short cuts if the study is to 


be a success. Rushing the study or applying controls without sufficient 


data will only end in negligible results or mediocre success. Perhaps 


It may be necessary 


the study cannot be completed during the first run. 
to continue the study the following week or the following month, but once 
it is properly completed and the necessary information gathered, properly 


analyzed and adequ@te controls established, it should be good forever more 


unless some assignable cause or variation creeps into the process. But if 


controls are properly established discrepancies will be quickly discovered 
and stand-by controls in the form of Average and Range (% & R) Charts 


can be promptly re-established. 


Perhaps there are faster or better ways to accomplish the results 


we have tried to achieve—later or more advanced techniques. Our main pur=- 


pose here has been to illustrate one way of accomplishing these results in 


@ manner that the operator and the shop foremen can understand, and the 
Plant Manager can interpret in dollar savings. 


"Chief Inspector, Solar Aircraft Company, Des Moines Plant. Senior Member, 


Iowa Section of American Society for Quality Control. 
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STATISTICAL QUALITY VONTROL IN THE FOUNDRY 


By George Ver Beke, 
Quality Control Engineer 
Union Melleable Iron Works cf Deere 2 Conpany, East Moline, Illinois 
Senior Member, Iowa Section, A.S.°.C. 


I imagine we have i.ad as difficult a time setting up 2 Quality 
Control Program as anyone. Wwe started about five or six years ago by intro- 
ducing Quality Control to two or three foremen in our plant by giving them 
a ten-day University course. After the completion of this ten-day course, 


they returned to their regular jobs of running 2 department. Naturally, 


these people did not have the time to spend to install Quality Control along 


nal 


with operating their departments. I assume that many df us have made the 
same mistake, treating Quality Control as a side line. We have found that 

to institute a Quality Control Program requires the full attention of whoever 
is installing the program. 

About three years ago, it was decided to see if Quality Control, 
properly installed, would operate in a foundry. I was at that time approached 
on taking a ten-day course at the University of Iowa on the subject called 
"Quality Control by Statistical Vetiods". I just could not refuse a two weeks 
vacation witl pay and a good expense account. fter two days at Iowa City, I 
was completely confused. ‘Why I ever thought this would be a two weeks! va- 
cation, I con't know. f£fter completing the course, I was very glad to return 
to work with my notebook full of notes which did not mean much to me. [I took 
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my notebook and put it in my desk, hoping never to have anything to do with 


it again. About the second morning after returning to work, the general sup- 
erintendent asked me what I thought of Quality Control. I could not let him 
know that he had wasted money dy sending me to Iowa City. I told him that I 
believed Quality Control was a good tool and would do us a lot of food. Eack 
in my own mind, I just could not see hon e college professor could tell how 
to operate a foundry with less scrap by having a fraction defective chart on 
each machine. It is true that we both had the same interest at heart - less 
scrap - but I could not for the life of me understand how we were going to 
do this. Now, in our foundry we Lave a great many experienced operators - 
foremen who heve been there thirty years, who have vecome exrerts on their 
particular jobs, and I did not thinks that I was capable of going out in the 
foundry and explaining to them how Quality Control was going to nelp them 
to reduce scrap. I knew that they would resent it. Actually, I would have, 
also. 

lie, as every other foundry, have tried different so-called bleck 
magics. ‘hen a salesman came in and told us he had a product to put in the 
sami to produce better castings wit. less scrap or produce better cores, 
we would naturally try some of his product. ‘We woula give it the fullest 
attention, and the results in the end would invariably be the same as we iad 
before. (je did rave the satisfaction of knowing that we kaa tried to reduce 
our scrap. Now don't misunderstand - we still believe thet in order to 
make advancements in the foundry industry, we should not be at 2 stand still. 
We should continue to try these so-called black magics. In the past, when 
trying these different products, the end result and effects derived would 
be strictly guess. Now, with Quality Control, we are able to prove whether 
or not we have advanced and how far. By using statistics, we are eliminating 


the guess work, 
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In July of 1948, I was called into the Manager's office. He ap- 
proached me on installing Quality Control in our plant by saying: "Now 
that you have the belief and the education, I would like to have you install 
such a program." It took me about a week to decide whether or not I would 
undertake this project. After deciding to accept this position, I then went 
to the John Deere Waterloo Tractor Works. I wanted to see the best appli- 
cation of Quality Control available in our organization. At that time, theiz 
Quality Control Program had been in operation over three years. I found that 
the X R charts were being used to a very large extent along with the P charts 
1 returned with the zealaf an Evangelist, determined to prove that Quality 
Control would work at the Union Malleable. 

We started our program by educating our inspectors and foremen, 
combined. Ernie Fay, who is the quality Control Adviser of Deere & Company, 
was our instructor. He taught us the purpose of the charts and the benefits 
which may be derived from their use, touching on mathematics as little as 
possible. We did not want to frighten our inspectors and foremen by intro- 
ducing them to a lot of theory. Mr. Fay also stressed the fact that we were 
to be of service to the foremen, and were not trying to take over their jobs 
or to run their departments. He also stressed the urgent need of coopera- 
tion between inspection and supervision. 

When this twenty-hour course had been completed, I still did not 


know how we were going to sample hot castings to the extent where we could 


find much scrap. 


It was decided to start in the core room. Good cores are one of 
the first steps to producing good castings. Our first chart was placed on 
the sandmuller, which mixes the sand for our coremakers. In the mixing of 
this sand, we use banksand, silica sand, corn flour, oil, kerosene, and water 
We had a specification of 3.5% moisture, In making our study, we used an 
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X R chart, checking each half hour. We found that the moisture ranged from 


1.5% to 6%. (Chart 1) We talked to our core room foreman as to what could 

be done to eliminate such a wide range. His answer was that, due to the bank- 
sand, we would never be able to do a better job of mixing without having a 

sand dryer, which would cost approximately $25,000.00. We then decided to 

check the moisture content of our banksand and found it to be 3% plus or mimus 1%. 


JOHN DEERE 


_ QUALITY CONTROL SAND 
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Moisture 


Chart 1 B Sand - Moisture 
We were deeply concerned and started checking the sequence of op- 
erations and the equipment used, The first thing we found’was that the water 
faucet. dripped all during the period of mixing, continually adding woisture 
to the sand. We had the water faucet fixed. The next thing we found was a 


two quart container which was badly beaten up. I doubt if it actually held 
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one quart. It was lined witi corm flour and oil, ecsuss 
lor measuring ail the in:redients. > rurchas * con= 
tainers to be used for each ingredient. “ec then found after these corrections 
that our second shift operator could maintein « 5% moisture, plus or minus 
e7%- The operator on the first shiit wa ; running 3.5% plus or minus 
2%. This continued lack of cooperation sulted in @ new operat« After 
we found the new operator operatin:; a plus or minus .7%, prov- 


the foremen that this *%25,000.00 


morning, a coremaker complained that his sand was 

the sand out of the hopper, took it back to the muller, did not do a tiling 
to it, and in about five minutes we returned the same sand to the coremaker, 
who began using it in about ten minutes, [I went to this coremaker, aski 

him how the moisture of his send was now. stated if we continued to pro-=- 
duce sand of this moisture content, he would never have any trouble making 
cores. Here is where Quality Control has tal.en the guess work out of the 
operation. 

After completing our setup as to sand moisture, our next venture 
was in our core pack-off. Having no inspection previously at this point, 
we deemed it necessary to install Quality Control charts at this time. Our 
core pack-off consists of six women. These women take the cores off the 
plates after baking, throw out the scrap, remove the fins from the cores, 
and then put the cores into boxes. These cores are then stored in the boxes 
until needed in the foundry. 


We started with a fraction defective chart, using e sample size of 


20, checking each half hour. This was purely visual inspection. After a 
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maintain proper core sand mix. . 
Just to prove that these coremekers could not tell the moisture in 

the sand, I will relate to you one experience which I had. This particular - 

—— 


couple of weeks, we found the quality of cores going to the foundry had 


greatly inproved. (Chart 2) 


DEERE SPEC. NO PART NAME 


CONTROL 
Percent _ 


N OPER 


Chart 2 Core Fack-Off 

We were being misled, because the foundry was continually running 
short of cores. This meant that one of two things was happening. Fither 
the coremaker was turning in more cores than were actually being produced, 
or we were throwing away too many good cores at the pack-off. We checired 
at the pack-oif on what these people were actually throwing out as scrap. 
we found at this point that it was here our shortage of cores was being 
created. What had hapnened is that these pack-off women hac become so great- 
ly interested in maintainine a desirable chart at 0% they were screprins 
cores which should not have been scrapped. We then nad to educate these 
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people on what was a defective core and what was a usable core. They were 
very cooperative, and imnediately we knew we were on the right track. The 
cifficulty of a core shortage in the foundry had been eliminated. The 

foundry foremen now often remark how much better the cores are as to quality, 
compared with cores previous to Quality Control. When, and if, we find faulty 
workmanship on any one check, we notify the foreman with a written reject 


notice. (Figure 1) 


JOHN DEERE QUALITY CONTROL 
REJECT NOTICE 


PART NAME MACHINE No. 


NOTIFICATION DATE AND TIME BY INSPECTOR 


+ FOREMAN 


FOREMAN_____ 


GENERAL 
. FOREMAN 


DIV. SUPT 


PROD. SUPT. 


M2868-6-49 Stock 


Figure 1 Reject Notice 
Returning 30 minutes later, we check the same operator, If the 
condition causing the scrap on this job has not been remedied, we issue a 
second reject notice to the foreman, returning again in 30 minutes. If 
this defective material is still being produced or passed, a third reject 
notice is issued to the foundry superintendent. “c, in turn, allow him one- 
half hour either to stop the job or correct the condition causing this scrap. 


If we find the same condition existing on the next check, a fourth reject 
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notice is written to the general superintendent. If steps are not taken to 
correct this condition in the next half hour, the fifth reject notice is 
issued to the general manager. To date, only six reject notices have 
reached the general manager. We use this reject notice system throughout 


the plant. At the beginning of cur program, we issued 70 to 50 reject 


notices per day. As of now, we issue approximately 10 a day. And believe 


me, the foreman does not want any reject notices issued to the foundry 
superintendent. He makes sure to correct a job that is going bad at the 
time of notification. 
After completing our setup on core pack-off, we decided to install 

Quality Control on our coremaking. Here again we had no inspection pre- 
viously. We hired a man for inspection who hed never seen the inside of 
the core room. He seemed very anxious to learn all about Quality Control. 
We trained him along the lines of inspection and what was a usable product. 
In about a month, we installed fraction defective charts on all our machines. 
We lave approximately 25 coreblowing machines. Using a sample size of 20, 

in looking for visual defects, we found thet the quality od cores was 
very poor. The coremakers began taking off scrap cores, rroduced a better 
quality core, and began to respect the Inspection Department. when we began 
our program, we had 60% of scrap being produced by the coremakers. This has 
been reduced to 1-1/2% as of now. There were several disagreements when we 
started our reject notices, as to whether the cores produced were of good 
quality, but after learning to live with each other and being able to decide 
correctly what was a usable core, our troubles ceased, We may now ask our 
core room foreman as to his opinion on Quality Control, and his ansver would 
ve "Don't ever take it out; it is helping me to do a better jct much easier. 
I am now having less trouble with the foundry foremer and least of 211 with 
my superiors." 


| 
. 
i 
- 
. ‘ 


After completing the core room program, we were ready to make our 
first move toward installing this tool in actual foundry practice; that is, 
in the molding and pouring of iron. We have a completely mechanized foundry 


using a duplexing system in melting iron. Our metal is melted in a cupola 


and is brought up to temperature of approximately 2850 in an air furnace. 


We have two molding units, referred to as units 1 and 2, operating two shifts. 
Fach unit is canprised of approximately 30 molders. Molding sand is conveyed 
to a large stationary hopper, located directly above each molding machine. 

The molds are placed on a roller conveyor, where they are held for pouring. 
We have six iron pourers continually pouring the molds produced by the 
molders. 

An investigation of past records indicated that a very large per- 
centage of our scrap was due to mis-runs. This investigation also revealed that 
No. 2 unit (day shift) was the greatest offender; therefore, we decided to 
make our first actual application in this division. 

There are many different jobs run on this unit each day; also, many 
molds vary as to the number of castings produced from the mold. Therefore, 
our sampling plan could be only an experiment. We decided to check about 
five (5) molds each hour, ‘Thus, the castings actually checked were five (5) 
times the number of castings in each mold. 

Fraction defective charts were used for each individual 
our first inspection consisted of looking for mis-runs and strains. The 
fraction defective in each sample was recorded on the chart end on the data 
sheet. ‘Ye not only recorded the fraction defective, but the actual reason 
for the rejection. 

We have taken the attitude that merely telling a foreman that he 
had 10% defective was not enough, but that we must tell him why the parts 


were defective, so he would know at once what corrective action should be 


We are now experimenting along different lines to enable us to 
inspect adequate portions of the castings produced. We check each job 
at least once an hour. Whenever we find any pattern operating above 0% 
mis-run, we notify the foreman, giving him a saiple of the defective cast- 
ing. He, in turn, shows the casting to the iron pourer. The inspector 
returns one half hour later to check the same job, to see if it has been 


corrected. If not, we begin using our reject notices in the same sequence 


OHN DEERE sree No. PART NAME PART NO 


QUALITY CONTROL Hard Tron Misrun 


Chart 3 Herd Iron Scrap - Mis-run 


In looking over our past scrap records, (Chart 3), our No. 2 unit 
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for mis-run, we found it to be 3.65%. Later this unit was operating at 070%, 
which proves that these professors did not have to know how to operate a foun- 
dry to enable them to educate us how to reduce scrap by using a P chart. We 
were all quite amazed by the results obtained by installing Quality Cmtrol 


on the pouring of iron. Ye had no idea that we could ever obtain a .70% 


° mis-run week after week. After completing No. 2 unit, we installed the same 

method of inspection on our No. 1 unit, and it has proved equally satisfactory. 
we were now ready to check for molders' defects, such as dirty, 

shifted, and strained castings. This is where we found our sampling to be a 
little difficult. Castings to be sampled are hot and dirty. “e check our 
castings within 15 minutes after pouring. You may well realize the condi- 
tion of these castings. It is rather difficult for an inspector to notice ie 
too many defects, which may occur unless he brushes the castings and has been 
trained and educated as to what defects actually are. We find it very im- 
portant to stay in close contact with our Hard Iron Department. This is 
the department that removes the sprues and gates from the castings; conse- 
quently, this is the first department that will catch scrap if scrap has 
been produced. Vie realize that due to the undesirable condition of the cast- 
ings in the foundry, we are bound to miss some of our defects which may occur, 
but by staying in close contact with the department which processes the 
castings immediately after cooling, we are, in many cases, able to help 


eliminate further production of scrap. Here again, we have a fraction defec- 


tive chart using the sample sizes, as we did for mis-run, for each operator. 
Using the same reject system, we have reduced our molders! loss greatly. 

One reason for excessive scrap in our foundry was because, previously, we 
had a post-mortem inspection, which generally was a day or two late. The job 
was completed, or we had made two days of scrap without knowing it. Ye are 


now able to notify the foreman of scrap within 30 minutes of pouring. The 
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total time elapse frou the making to the inspection is about 30 minutes. 
I think you will realize the importarce of this sort of inspection setup. 

In February 199, we had completed our setup of Quality Control in 
the molding and pouring section of the foundry and were now ready to set up 
2 Similar rrogram in our Fard Iron Department. 

This department consists of about 35 people - two shifts - whose 
duties are to remove sprues and gates from the castings, chip off all core 
fins, throw out all scrap, and load the accepted castings into annealing 
pots. We also grind our lighter jobs while in the hard iron stage. We refer 
to these people as chippers and grinders. We have both men and women doing this 
sort of works 

Previously we had four inspectors who were referred to as "roving 
inspectors". The duties of these inspectors were more or less to police 
the department as to the quality of work. I now think a better name would 
have been "wondering inspectors", because all they actually did was wonder 
why the good Lord made the day so long. When it came to inspection, there 
was very little done. 

We also had four scrap counters who were responsible for counting 
scrap castings thrown out by the chirppers. Actually, all these people did 
was count the number of castings that had been delivered to them from the 
chippers, ignoring whether the castings were scrap or salvable. In other 


words, we were throwing away good castings. At this point, we started to 


absorb the people from normal inspection. We took two of these inspectors 


into the office and explained to them that they were to become Quality Con- 
trol Inspectors. 

In order to secure the full benefit from our new setup, we placed 
a fraction defective chart on each chipper and grinder. The inspectors 


would check a sample from a pot, to see that all the necessary operations 
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hed been performed and if scrap castings were being thrown out. If one de- 


fective piece was found in the sample a second sample was taken immediately. 


If any defectives were found in this second sauple, the pot was rejected and 
reclained by the operator. at the start 15 to 20 pots per day were being 
rejected. This resulted in several misunderstandings, and grievances were 
filed by the union, the operators feeling that we were being unfair to them. 
They did not realize that what actually was wanted was a thorough job of 
chipping with all scrap removed. It is true that it was necessary to retime 
some jobs, because of additional work that was added. However, this was a 
small percentage, and most jobs were timed to give us a quality product. At 
the present time, we seldom reject over one or two pots a day. 

After the hard iron inspectors were trained and all machines charted, 
we decided a better job of inspection could be attained, by eliminating the 
four scrap counters, and by having inspectors count all the scrap at the 
time of our sampling inspection. This idea was not accented with much en- 
thusiasm b7 the inspectors, but after a little convincing and actual prac- 
tice, the inspectors welcomed their new duties. This setup enabled the in- 
spectors to see all the scrap; whenever salvable castings had been placed 
in the scrap hopper, the inspectors were able to return the castings to the 
operator for salvage immediately, thus eliminating unnecessary scrap. Another 
thing we found by having the inspector count the scrap at the time of opera- 
tion was trat we were able to notify the proper authorities on how much scrap 
we found on any particular job and for what reason. Consequently, we en- 
abled them to prevent the same condition from continuing. We have one 
inspector, whose duty is to work between our hard iron inspectors and our 
foundry inspectors. This enables us to relay information missed in the 
foundry to the inspectors who are responsible for these particular jobs. 

Just recently we have completed our program in the Finishing Department. 
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This devartment consists of four sections. The first section is the clean- 
ing section. Here all castings are cleaned either by milling or sand-blasting 
after annealins. fter cleaning, the castings are put into pans whici. hold 
approximately 2000 pounds. These pans are then weigh-counted and delivered 
to the grinding room. Here, again, we had roving inspectors alonz with 
scrap counters. We trained the normal inspectors in Quality Control, and 
placed P charts on each grinder. We check each hour to see that castings 
are properly ground and that the scrap is being thrown out. By having tle 
inspectors count the scrap, we were able again to eliminate the scrap coun- 
ters. The inspector is also responsible for checking each load before it 
leaves the department. He places a small tag which he has signed on each 
load, signifying that this load may be movea to the next department, 

The third section of this department is called the bench room. Here 
castings are straightened either by a drop hammer, hydraulic press, or manual 
labor. A considerable amount of air chiseling of webs and drifting of holes 
is also done. The setup here is the same as in the first two sections. One 
inspector handles the department, releasing material by using the tag system. 

The fourth section is our drill room. Here are located all drill 
presses, tapping machines, and punch presses. There are about 25 people 
working in this department. Checks are made on an hourly basis, and a Pp 
chart has been placed on each machine. Go and no-go gages are used on the 
majority of operations. 


Wwe are checking the operators for quality of work produced and for 


removal of all scrap from the load. Again, in checkins our samples, if we 


find one defective, we take a second sample; if we find one or more defective 
pieces the second time, we reject the load for reclaiming by the operator, 
This inspector is responsible for all loads leaving his department, because 


this is the last department prior to shipping. He places a green tag on the 


load if ready for shipment. A red tag is used if load is not nrepedty com= 
pleted or cperations are missed. 

About two months ago, we started to check all of our outgoing mater- 
ial by sampling each load to be shipped. We use the techniques of the Army 
Crdnance Tables as a basis for sampling to determine whether the load is to 
be accepted or rejected. We have noticed marked improvement in the quality 
of outgoing material. Our returns and complaints have decreased considerably. 
In one of our plants, our quality of work was running 5% defective. ie checl.ed 
with this plant about two weeks ago and found the percent defective less than 
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» point out the reduction in scran, we refer to Chart 4, whioh 
gives the ratio cf scrap to total produced, with the monthly and yearly; 
frox 1916 through part of 1950. 
Since Quality Control was started in 1shisé, and time wes given to tzke 


corrective acticn, and for cperators to become familiar with the program, it 


will be noted that a continued dowmward trend in percentage of screp has oc- 


curred; in the past few months less than 8%, whereas before Quality UCcntrol, 
the best year was 13%. Furthermore, as long as the trend continues downward 
cre is a probability that the average will be still lower. 

In order to operate the Quality Control Frogram, it was necessary 
to add two people to the inspection staff. However, this addition has maue 
it possible for us to make inspections in the core room and foundry proper, 
vhere previously no inspections were made. 

About five months ago, we decided that we were ready to apply 
Quality Control in the Vermilion Malleable Iron Yorks, another subsidiarn 
of Deere & Company. This foundry is of a batch type, in which we have 
about 35 molds, who pour their own molds twice daily. 

This foundry presented an entirely different problem than we had 
previously encountered. Therefore, considerable thought and study were re- 
quired before we adopted any sampling methods, as we certainly did not want 
to upset any of the people, either workmen or supervisors, but we did want 
to get information relative to defects during the same day the castings 
were wade, and not have to wait two or three days before we knew what quality 
was being produced. 

We have four inspectors who check after each reat or time oi pour- 
ing by dumping aprroximately 10% of the molds wade by cach molcer, and then 
knocking off the sand and brushing the castings, so they are able to inspect 
these castings for various defects. After gatherins this inZormetion, it 
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is plotted on a percent defective chart, which we call a molding machine 
chart. The inspector also carries and makes out an additional data sheet, 

on which he records the same information as is recorded on the chart and 

its data sheet. After all floors have been checked and all machine charts 
plotted, the additional data sheet is given to the foreman, who examines 

eack individual job reported with any percent defective by an examination 

of the individual castings, with a view of determining the reason for the 
defect. The foreman then goes to each molder who has scrap and explains to 
him the percent defective found in his castings and the reason. This informa- 
tion is generally given within one hour after pouring, 

After a few weeks, we found we had reduced the scrap considerably. 
Mis-runs seemed to account for about 0% of the scrap, or about 2% of the 
total castings made, and it indicated to us that we had not reduced this 
kind of defect very much. 

We then picked out the 10 molders with the highest percent of re- 
ject due to mis-runs, and assigned two inspectors who were to check two molds 


out of each 7 or 8 molds at the time of pouring. Whenever one piece of mis- 


run was found in any sample, the inspector immediately notified the foreman, 


who, in turn, went to the molder and informed him of various means of elin- 
inating mis-runs on that particular job. 

We nad found that at the beginning of a pouring period we had a 
greater amount of scrap than later on during the period, and investigation 
revealed that undoubtedly this was largely due to cold ladles. 

After about two weeks! operation with the above plan, we found 
that we had reduced our mis-runs from 2% to approximately .75% and seemingly 
we are able to hold that level. 

We are now in the process of insta!linz pre-heating devices for 
our ladles, which, we believe, will assist us further in lowering our percent 
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defects per mis-runs. 

In this smaller foundry,we have found conditions entirely differ- 
ent from those in the other foundries where we had installed cuality Control, 
in that a greater degree of cooperation already existed. 


Prior to our investigation and the installation of Quality Control, 


this foundry was producing about 8% scrap, and after three months of inforn- 


ation presented through Quality Control procedures, the scrap had reduced to 
less than 5%. 

Reject notices are not needed in this foundry, because supervision 
takes immediate action as soon as they see a bad plotting, or as soon as the 
Quality Control man informs them of bad conditions. 

As a further aid to keeping our supervision and operators informed, 
we have installed a large poster board, which displays each molder's name, 
together with his percent mis-run, miscellaneous, and total scrap. Also, on 
this board we have the rated position of each molder in the foundry, this 
being the actual standing so far as his percent scrap is concerned. ‘le have 
found that these molders are very conscientious about their standing or 
rating and their own relative position with respect to the rest of the mold- 
ers. 

At the end of each week, we place a gold star after each of the 
five molders wi.o have the smallest amount of scrap for the week, and this 
has proved quite effective, for it is obvious that no molder is very happy 
when his name appears at or near the bottom of the list. 

The employees and supervision of this plant have accepted Quality 
Control as the useful tool and have shown splendid cooperation in the intent 


to produce better quality castings with less scrap. 
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ROLE OF STATISTICAL QUALITY CONTROL 
IN AIR FORCE PROCUREMENT 


INTRODUCTION = The “Aircraft Nails" 


In speaking of the "Role of Statistical Quality Control in Air Force 
Procurement" I feel that it will be more interesting to you, and, at the 
same time easier for me, if I build this discussion around a specific item 
of procurement which has long been a source of inspection difficulty, and 
use this example to illustrate those policies and procedures which collec- 
tively we call "Statistical Quality Control." The example that I intend 
to use is the case of the aircraft nail industry which, as far as the Air 
Force is concerned, is represented by five major producers. In order to 
respect their enittiinnen, we will call them the Jack, Queer, King, Ace and 
Deuce Nail Companies, Now before any one asks what aircraft nails are I 
confess that the term "aircraft nail" is a pseudonym for an item which is 
manvfactured in immense quantities and is of great importance to the Air 
Force. Except for the identification of the item itself, the facts which 
I shell present are completely authentic. 


AMC Crganization and Quality Control 


let me digress for a moment to present a brief picture of tne cverall 


organization within which we are trying to promote the use of statistical 


techniques, The details of organization are too extensive ani too dry to 
discuss on this occasion, let me just say that the Air Materiel Command is 
basically divided into three directorates; Supply and Maintenance, Research 
and Development, ard Procurement and Industrial Planning. The Directorate 
of Procurement and Industrial Planning is itself composed of three Divisions; 


Procurement, Industrial Planning, and Quality Control; the latter concerns 
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itself with the evaluation of the quality of all material procured by the 
Air Force, 


The AMC's Job = Its Scope 


You can get some idea of the immensity of the AMC's procurement and 
inspection problems when I tell you that during the fiscal year 199 the 
Air Force through theAMC procured about two billions of dollars worth of 
material of one sort or another, and does business with an average of 6500 


contractors per year. In the New York procurement field office alone there 


are 1200 active contracts per month; speaking in rough averages 200 con- 


tracts expire and 200 contracts are instituted per month, It is clear that 


the Air Force exercises a profound effect on thousands of businesses and tens 


of thousands of people. Obviously, then, when the numbers with which we are 


dealing are so large, it is imperative that the Government in both its in- 
spection and acceptance functions be fair to its vendors as well as to itself. 


Surveillance Inspection Policy 


Statistical Quality Control is, of course, concerned with developing and 


applying procedures to achieve such fairness, But before we discuss the de=- 


tails of SQC it is necessary to mention briefly the Quality Control Division's 


Surveillance Inspection Policy. It is basic to our discussion. By "Surveil- 


lance Inspection" we simply mean that inspection is actually performed by 


contractors at the direction of the Government, 


In other words, instead of 


attempting to perform inspection itself, the Government assures itself that 


the contractors' inspection system is adequate, This assurance is based on 
a thorough inspection of the contractor's methods, tools, gages, organization, 
: and so forth. If such is the case, it follows that the inspection performed 


by this system will be satisfactory provided that the system itself remains 
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unchanged, a condition which the air Force inspector assures by periodic 
follow up surveys. In short, we can say that the sir Force chooses to 
inspect not the material it is receiving but rather the inspection system 
the contractor has provided to insure that only acceptable products are 
delivered. This policy is based largely on practical necessity. The most 
casual investigation will reveal that it is manifestly impossible for the 
14,00 odd air Force inspectors to inspect physically all the material that 
the Air Force procures. By the same token, if the air Force were to hire 
enough inspectors to do that sort of a job, the cost would not only be 
prohibitive, but since it would require an almost exact duplication of 

of the contractor's inspection systems it would be grossly inefficient. 
But enough of this background discussion = let us return to the case of 
the nail industry. Incidentally throughout this paper I am presuming that 


contracts for purchases are let on the basis of competitive bidding. 


Aircraft Nails - Quality Low - Inspection Costs High 


Approximately six months ago the Statistical Quality Control Unit, 
which was just becoming operational, was asked to review the acceptance 
inspection procedures for (and the quality of} aircraft nails. A very 
comprehensive study indicated that the entire situation was unsatisfactory 
both to the Air Force and to the contractors. It was quite apparent that 
the Air Force was receiving inadequate inspection protection because the 
specification for nails was quite flimsy both because of what was and was 
not required. At the same time the contractors were unhappy for two rea- 
sons; first, quality evaluation methods were not standard and thus accept- 


ance of material depended as much on subjective as objective criteria; 
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(this, of course, placed some contractors - even the best - in a less favor- 


able competitive position than others). Secondly, the cost of inspection 
was excessive; in fact it cost more to inspect the nails than to manufacture 
them. 


The Specification 


Now, having discovered that there was much to be done, we felt that 


the specification was the major root of all this evil. So lets take a look 
at a portion of the applicable specification (Figure I) and study the accept- 


ance requirements for dimensional characteristics only. 


Extract of Acceptance Procedure for "Aircraft Nails" 


F=-5. Sampling.- 
F-5a. Finish and Dimensions.- Sampling for finish and dimensions 
shall be random in accordance with Table VI. The acceptance-rejec- 


tion provisions of Table VI only shall apply separately to each of 


the inspection elements or requirements. For example in the case of 


dimensions, two of the inspection elements are over-all length and 


shank diameter. 


TABLE VI 
Sampling for Finish and Dimensions 


Combined Samples 
Lot Size : Sample : Sample: acceptance No. : Rejection No. 
Size : 


: 
: 
3 
Under 25 : First : All: 3 
26-75 : First 25 3 1 
: First 25 3 0 5 
: Second; 50 2 3 
: First =: 50 3 1 : 4 
901-5000 : 100 3 : 
000 Second 150 3: 5 6 
: First 2 6 
Over 20,000 : Second ; 200 ; 5 : 6 
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Let us suppose we are nail manufacturers and that we are thoroughly scruti- 
nizing the specification to determine to what quality standards our nails 
must conform to be acceptable to the Air Force. The fact that a sampling 


plan is written into the specification is in itself an important fact; the 


Government indicates thereby that it does not require evidence that the 


nails are perfect or even near perfect. So now we ask - To what degree 
may nails be nonconforming and still be acceptable? Paragraph F-5a,(Figure 
I)of the specification partially answers this question. You will note that 
this paragraph says that Table VI applies separately to each inspection 
element. That word "separately" is extremely important. A question may 
now be proposed "What percentage of nails may be too long and still be 
acceptable?" Lets look at Table VI and see what that tells us. Assume 
that our nails are made in three different lot sizes - N = 1,00, N = 1,000, 
N = 25,000. We want to know what are the chances that a series of lots of 
any given size will be accepted if the percent defective for length is 1%, 
2%, 5%, etc. By preparing operating characteristic curves for each of the 
sampling plans in Table VI we can estimate our chances of acceptance (Pa) 
for any given percent defective (p). The operating characteristic curves 
(Figure II) are very revealing. I need not comment on them in detail but 
you will notice that for the two larger lot sizes material approximately 
1% defective is acceptable 95% of the time; however the smaller lot 

(76 - 100) are accepted only 75% of the time at 1% defective. Of course, 
the knowing manufacturer will waste no time in discovering that the lot 
size has an important bearing on acceptance. You can ponder the numerous 
implications - economic or otherwise - of these facts. But lets not stop 
here. Suppose the nail has 10 dimensions. What then? Does this mean that 
you could accept nails as bad as 10% defective? A survey made by the AMC 


has shown that the quality of the nails accepted by the Air Force reflects 
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the fact that the specification allows a rather liberal AQL for each 
characteristic. 


Revision of the Specification 


Obviously, if we are to do anything to improve the situation we must 
first direct our attention to the specification and define as clearly as 
possible the procedures by which material is judged to be acceptable or 
non=acceptable. This is, of course, a sizeable task because a sound in= 
spection program cuts across both inspection by attributes and by variables, 
as well as numerous problems in sampling for various physical tests, de- 
structive or otherwise. However to simplify this discussion let us restrict 
the scope of our problem to revising the specification only for inspection 
by attributes, i.e., for finish and dimensions, as mentioned earlier. For- 
tunately the Military Establishment has a standard which provides numerous 
sampling plans of considerable flexibility which can be used for inspecting 
nails and with considerably greater effectiveness than the plan we men- 
tioned previously. This is JAN-STD-105; it is mandatory for use by the 
Armed Services for sampling inspection by attributes. Therefore the first 
measure we will take to improve the present inspection specifications for 
nails is to reference JAN-STD-105. This Standard is exclusively a group 
of Sampling Tables so, for a moment, let us see what happens when Ace 
Company presents for acceptance a series of lots varying in size from 3200 
to 8000 units each. Assume that these lots are uniformly 5% defective. 
What fate awaits Ace's nails? All we need to do is consult, in cook-book 
fashion, JAN-STD-105 and we find that the 0.C. Curve for the applicable 


sampling plan for these lots show that material 5% defective will be 


accepted 95% of the time (Figure III). But you say, "Suppose the Ace 


Company presents &%' material; what then"? Well, looking at the 0.C. Curve 
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we find that lots of that quality will be accepted about 0 percent of the 
time. But is the Air Force willing to accept lots that bad and if not 

what will the AF do? The answer is that when we determine that quality is 
significantly worse than the AQL we will put in a more stringent inspection 
plan by which material &% defective will be accepted only about 5% of the 
time and 5% material will be accepted about 55% of the time. (Figure IV). 

The conditions under which such "tightened inspection" is applied is well 
defined in JAN-STD-105. Likewise, under certain conditions - when the con- 
tractor produces particularly good material - reduced inspection may be used. 


(Figure V). The attached operating characteristic curves, if carefully 


studied, indicate the various features of normal, reduced, and tightened 


inspection. 


Quality Standards 


Now I know that some of you are asking how we "define clearly" the 
quality requirements of the Air Force for inspection by attributes. The 
question really is: “How do we implement JAN-STD-1057?" The answer is that 
the Air Force establishes for each inspection item applicable Acceptable 
Quality Levels and also specifies the quality characteristics (Classifica- 
tion of Defects) for which inspection is required. Material which (for a 
class of characteristics) is not worse than an established Acceptable Qual- 
ity Level will be accepted 95% of the time. The quality characteristics are 
grouped into different categories of seriousness by engineers who base their 
judgments on arbitrary criteria fixed bg the Air Force as follows: 


DEFINITIONS OF DEFECT CLASSES 
Critical Defects 
A critical defect is one that judgment and experience indi- 


cate could result in hazardous or unsafe conditions for 
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personnel using or maintaining the product; or, for major- 
end products, such as ships, aircraft, or tanks, a defect 
that could prevent the unit of product from performing its 
tactical function. 

Major Defects 
A major defect is a defect, other than critical, that could 
result in failure or materially reduce the usability of the 
unit of product. 

Minor A 
A Minor A defect is one that reduces the usability of the 
unit of product slightly. 

Minor B 
A Minor B defect is one that does not reduce the usability 
of the unit of product but nevertheless is a departure from 
specifications or drawings. 


Consumer — Producer Relations 


So far, then, we have outlined a plan which establishes clearly the re- 
lationship between the Government and the vendor insofar as quality require- 
ments are concerned. As far as possible we have defined our quality re- 
quirements in quantitative terms because we feel that both the consumer and 
the producer are entitled to know the inspection hazards to which they are 
exposed. Producers require assurances that acceptance will not depend on 


the whims and fancies of the consumer; that all competitors, whether in 


Boston or Los Angeles, will be treated alike; and that the costs of quality 


evaluation will have some rational relationship to the importance of the 
item and to its cost of fabrication. Likewise the consumer should not be 


at the mercy of the producer. Enough of that, but before we return to the 


& 


Ace Company's nails it may be well to ask the jackpot question, "What is a 
fair AQL?". 


Determination of AQL 


let us answer this question within the frame work of present Air Force 


procurement policy, i.e., there is no tie in between cost and AQL. In 
other words the contractor is responsible for delivering 100% acceptable 
material but the Government for practical reasons is willing to accept less 


than 100% quality. To the technical man it would appear that this problem 


is no problem at all. 


All you have to do, says he, is ask the aircraft 


nail designer just how much defectiveness can be tolerated and set the AQL 


there. Unfortunately we run into a particularly perverse breed of cat when 


we confront the aircraft nail engineer. All one gets is a lofty declara- 
tion that aircraft nails go into airplanes, airplanes fly, people get 


killed when they suddenly stop flying; therefore all aircraft nails must 


be perfect. It avails nothing to say that nails have been averaging 10% 


defective for years or that "perfection" of any sort is economically, if not 


physically, impossible. Next you might say, "Well, what can the industry 


economically produce? We will set the AQL there." The logic of this ap- 


proach appears sound but it too has its weak points. First, reliable in- 


formation regarding the quality capability of an industry is not usually 


available and is quite expensive to obtain. Secondly, it is difficult to 


say what the "average" quality of an industry really is without compromis- 
ing the interests of manufacturers who have records of particularly high 
quality production. 


In the case of the aircraft nail we conducted a pro=~ 
cess average survey to determine exactly what the nail industry is now doing 
quality-wise. 


This survey was simply a study to determine the proportion 


of defectives in samples of nails taken from representative lots from each 
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of the cooperating manufacturers. Without going into the methods of con- 
ducting this survey here is what we learned at the Jack, Queen, King Ace 
and Deuce Aircraft Nail Companies: 
FROCESS AVERAGE SURVEY 
for 


AIRCRAFT NAILS 


QUALITY (PERCENT DEFECTIVE) OF MATERIAL 
PRESENTED FOR ACCEPTANCE 


MINQR=a 
A 


0.6 
25 
1.1 
0.3 
11.3 


QUEEN 


KING 


ACE 


ce ee ice se ice seiee 


B 
2 
2 
2 


se se ise se 


DUCE 


NOTE: Column A is percent defective. 
Column B is number of characteristics inspected. 


FIGURE VI 


This survey was based on inspection for those characteristics listed 
on the C/D. However to present added inspection costs being charged to 
the Government, contractors were requested to report the incidence of defec- 
tiveness only for those characteristics which were, by standard practice, 
subject to Government and contractor inspection. (The AQL’s finally pro- 
posed for the nail industry were 5% for Major characteristics and 6% for 
Minor A. A reasonable interim period is allowed for the contractor to 


make necessary adjustments to meet these AQL's). 
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The AMC Inspection Plan 


It is well to pause now and see what kind of a plan we are offering 


the Ace Company. Here it is in outlines 


Item = 


Iot Sizes = 


Quality Charace- 
teristics 


Requiring 
Tnspection ~ 
Tests 
ssociate 
sampli 
Procedures — 


Sampling Plan - 


AQL's = 


THE AMC PLAN 


Aircraft Nails 
Optional, provided material is homogeneous 
to the best knowledge of the contractor and 


Government. 


Classification of Defects 


(Qnitted from this paper) 
JAN=-STD=105 (For sampling inspection by 
attributes ) 


(Applicable to classes of characteristics) 


5% Majors 
6% Minor A 
Minor B 


(No critical defects listed) 


NOTE: Usually the details of an inspection plan are not 


included in specifications. Instead JAN-STD=105 is 


referenced and a standard inspection plan is prepared 


to implement the standard. 


The Ace Company had little or no difficulty in meeting our AQL'ss; the 


Classification of Defects was found quite reasonable. There was only one 


real complaint against this plan; while it was protective, objective, and 


standard, it was still too expensive. It cost more to inspect aircraft 
nails than to make them, a state of affairs that brings us to the sub- 
ject of process control, 


Frocess Control 


This discussion has so far been restricted to problems of acceptance 


sampling which simply poses these questions: What is the quality of this 


material? Should I accept or reject it? There is, as you are well aware, 


another area of SQC called process control. This is by far the most fruite 


ful field of Quality Control and in fact, to most production and industrial 


engineers, it is SQC. Unfortunately, statistical process control is not 
entirely under the scope of what the Air Force can consider its proper 
business. Without dwelling on this point at length, we can say that the 
Air Force is primarily a customer and, as such, should not concern itself 
with matters that are essentially in the field of production control, i.e., 


process control. 


The Air Force Position on Process Controls 


However, process control does present numerous problems because there 


is a tendency for manufacturers to consider process control in itself 


adequate protection for the Air Force against the acceptance of sub stand= 
ard material. The air Force, of course, wants to encourage the use of 
sound statistical process control methods but it would take a legion of 


engineers and statisticians to determine when a particular process control 


system actually gives the Air Force the protection it requires. To date 


we have steered a middle path by reducing acceptance sampling to a minimum 


when a process is known to be under effective controls, The term "to a 
minimum” means that we not only use "reduced" inspection as mentioned 


previously but on the basis of process control evidence we plan to group 
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lots into a "grand lot" and then sample this very sizeable population. 

In this way, of course, we cut our inspection requirements tremendously 
but at the same time we expose both the Air Force and the manufacturer 

to the hazard of rejection of an immense quantity of material. This 
problem becomes further complicated when the initial sub-lots of a grand 
lot are shipped before inspection of the grand lot is completed. Statis- 
tical methods, however, are available to minimize the hazards involved 
in a plan of this kind. 

Results of an SQC Inspection Plan 


SQc pays off when a manufacturer is truly producing under controlled 
conditions, A good example of how SQC thinking benefits both the Govern- 
ment and the contractor is the case of the ACE aircraft Nail Company. 
Shortly after our original process average survey was made, this company 
established a complete statistical quality control organization. This 
new unit proceeded to institute statistical process controls practically 
on a blanket basis throughout the plant. Iwill lea’e it to you to judge 
the effectiveness of this program from the "before and after" results for 
nails outlined below: 


BEFORE PROCESS AFTER PROCESS 
CONTROL CONTROL 


FIRST CHECK SECOND CHECK 


MAJ. MIN-A MIN-B WAS. WIN-A WIN-B : WAY. WIN-A WTN-5 


Defective Defective Detective 
24.0 11.3 NO Ob Ooh 0.3 0.0 0.8 
2.5 INSPEC 0.5 0.2 0.2 0.8 


12.7 8.3 TION 1.3 0.C 067 NONE MANUFACTURED 


FIGURE VII 


AIRCRAFT 2 
NAILS 2 
TYPE 
A 
B 
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Qn the basis of this data, the Air Force authorized a "grand lot" 
evaluation plan for this company, The final result is of course that 
inspection costs have been cut significantly and production efficiency 
has reached a new high because operators anticipate "out of tolerance" 
trends in their machines. Quality Control thus achieves a positive 
effect. Evaluation is accurate, objective and economical; statistical 
controls are so sensitive that it is possible to adjust manufacturing 
processes before trouble develops. The further details of administra- 
tion, cost adjustments, statistical methods, and revisions of specifica- 
tions are so numerous that it seems best to leave it to you to choose 
those of greatest interest for discussion during the question periods 


Conclusion = Resume 


The function of SQC in Air Force Procurement is to assure the Air 
Force against the acceptance of sub-standard material. The methods by 
which such assurance is realized are based on numerous considerations of 
statistics, engineering and costs of inspection, The objective of SQC 
4s to measure the quality of material by techniques which (1) make it 
possible for both the contractor and the air Force to know their respec=- 
tive inspection risks,(2) identify specifically the quality characteris- 
tics which, for engineering reasons, are the foci of inspection, and (3) 
permits evaluation of quality at minimum cost, To the contractor this 
means (1) that in the future the measure of quality will be in more pre~ 
cise engineering and statistical language. (This, in all likelihood, will 
have an important bearing on competition). (2) Inspection, as a part of 
the overall SQC organization, will take on a positive function in manufac- 
turing. (3) As far as the air Force contracts are concerned, sound 


sampling inspection procedures will be written into specifications to 
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replace present nebulous inspection provisions. However statistical pro- 
cess control will not be considered to be of direct concern to the Air 
Force, but it may be an important factor in determining the severity of 
final acceptance inspection requirements. 

As a concluding thought I should like to caution you that this review 
of the "Role of SQC in Air Force Procurement" makes SQC sound somewhat 
more "grown up" than it really is. To us, SQC is simply the common sense 
application of accepted statistical and engineering methods to the solution 


of problems of quality evaluation and control. However the job of revising 


specifications, preparing standard inspection plans, integrating process 


- control and acceptance sampling, devising sampling plans for non=attribute 
inspection and tests, not to mention the job of "selling" SQC now and then, 
is go gigantic that we have only nibbled at the edge of the complete SQC 
program. But we can assure you that the Air Force is thoroughly quality 
conscious and is making an all out effort to use techniques for measuring 
quality which are in keeping with the scientific spirit of our times, 


t 
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Measurement and Control of Flavor Quality 
David Re Peryam 
Quartermaster Food and Container Institute 
Chicago, Illinois 

Quality control oan apply to almost anything that is manufactured, at 
least wherever it is possible to evaluate in terms of good and less goods 
Whether statistical quality control oan apply so universally I do not know, 
Perhaps the common practices of S.Q.Ce, as we see it evidenced today, can 
not be transferred with efficiency to some industries or to certain oriti- 
cal aspects of their production. But it is becoming more and more evident 
that the basio idea of S.Q.C., that of practical use of scientifically 
sound measurement, has a very broad significance. Everyone may not find 
his problem fully solved therein, may not discover that the detailed 
methods he needs are fully matured and already made available by the 
pioneers of S.Q,C.3 but he can find within its philosophy encouragement 
to tackle his own individual problem as well as demonstration of what the 
most valuable approaches are likely to be. 

The field in which I work bears suoh a relationship to this new 
science. Through academic training, interest, plus the fortunes of 
employment I have always found myself concerned in production of food or 
related materials, and for the most part in fields related to quality. 
When I first began to hear about S.Q.Ce I sensed an exciting parallel 
between the problems it handled with such aplomb and those with which I 


wrestled. I suppose my real feeling was, "Yes it's beautiful, scientific, 


and tempting, but how can that mathematical stuff do us any good here?" 


Yet upon my continued exposure to its ideas and its works I came 
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to appreciate the fact that I was already in the field without knowing 


ite That I am asked to participate here at the clinic sessions indicates 


to me that the professional S.Q,C. men concur in my belief. 


Now for the moment I shall seem to digress. You may think I have 


become a bit elementary and obvicus, but I want to make a point. When 
we talk about "quality" for quality control purposes, in spite of the 


many interpretations that might be given the concept, we know pretty 


well what is meant. Most interpretations cf quality resolve into a 


matter of utility. 


If a product appears completely suitable for the pur- 


pose for which it is intended, by labelling it “good quality" we predict 


that it will give satisfaction to the ultimate consumer. Often the 


Quality Control man is lucky and oan have very definite, if perhaps 


arbitrary, standards for what good quality shall be. Again, quality 
control may be required to discover within the production process itself 
characteristics in the product which are critioal for this prediction of 


utility and, further, may have to devise ways to measure them reliably 
and efficiently. 


In all this the availability of methods of measurement 


is of vast importance. Consequently S.Q.C. has found most ready acocept- 


ance and its most precise application in those industries and for those 


processes where strictly objective measurement is possible, 


Now consider production of food, beverages, and like products. Here 
we will find many aspects of quality that are accessible to objective 


measurement and therefore should also be subject to quality oontrol by 


use of standard techniques. In this category will fall such factors as 


bacteriological and chemical purity, adherence to recipes and physical 
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specifications, and conformity to arbitrary physical grading standards, 
Also, wherever you have mass production it is usually possible to estab- 
lish interim controls on the processes as such, making the reasonable 
assumption that such controls will somehow carry through in the quality 
of the product. This, of course, applies to food also. 


But there is a facet of the quality of foods and like products 
where measurement is difficult. Moreover, this aspect is frequently, if 
not always, in the commanding position when it comes to determining 


total quality. This is the factor which may be called, simply, flavor. 


If a food is to be good the effect of its flavor mst be positive, i.e., 


pleasant, or at least it mst not have a negative effect. Yet as soon 
as we say "flavor" and admit that it is something more than & combination 
of chemical substances that impart smell and taste, we are in a new realm 
Flavor is a concept that has meaning only in relation to the human senses 


for it arises in the interaction of the senses with the chemical environ- 


ment, Therefore, its measurement mst be in the same terms. Many factors 


which are subject to control in process by way of measurements which are 


strictly objective and precise contribute to flavor and their control 


represents partial flavor control. But a final evaluation of flavor 
mist be in terms of human behavior. 


Here is where quality control rums aground. Food producers can 


control processes, they oan control properties related to flavor, but 


when it comes to the crucial aspect of the flavor itself, they are at a 


loss. Not that it is hard to know and evaluate flavor. 


Everyone can 
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and does do this but only on a personal basis. There is moh vari- 
ability in both perception and appreciation of flavor and every man has 

a tendency to feel that he is a judge unto himself. And so he is = for 
himself alone. But the manufacturer oan not afford to define quality in 
terms of the responses of a few persons. Evaluating flavor so as to take 
account of the probable reaction of the consumer en masse can not be 
done in this easy, naive way. To predict mass behavior, we mst sample 
behavior, we mst measure it by rigorous, standard procedures, and we 
must be able to understand the significance of what we have done. 

Among food processors and research workers attitudes toward the 
problem of flavor control vary from the supremely confident to the 
apathetic. The former goes with the belief that individual judgment is 
entirely sufficient, so we do not need a careful approach; the latter 
with the belief that the problem is not oapable of solution, so we should 
not waste time on it. That both attitudes have arisen is understandable 


in light of the present situation. Which situation can best be described 


in a flat statement: A standard methodology for dealing with flavor 


problems does not existe 


However, such methodology is possible. It is even now being devel- 
oped by some middle-of-the-roaders; yet it is still poorly formlated. 
Also, knowledge of even what little has been done is not widespread. 
Until it is flavor control will not be generally scientific and effective. 

Tonight I propose to tell you something of what is being done in 


this field. First, I will present in some detail what is undoubtedly 
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the best example of scientific flavor oontrol in industry today. Then 
I will discuss some experimental work that is suggestive of the way 


further progress is to be made. 
That best example of flavor control is the system used by the 


Seagrem-Calvert Distilleries. Their production organization changed 


over to a scientific approach to flavor problems about sleven years ago 
and has used the present system about eight years. I was with their 
quality Laboratory for four years during the period when the method was 
being developed and installed, and came to know it first-hand. Since I 
left the organization and have had an opportunity to learn more of flavor 
control problems and of their solutions among food producers at large, 

I have come to appreciate the effectiveness of the Seagram-Calvert 
methods. 

At each of the company's opereting plants there has been established 
an organization which is designated The Quality Laboratory. These are 
essentially psychology laboratories rather than whiskey laboratories; 
their basic procedures are determined by that science and their function 
is to use human observers in development and process control work and, 
finally, to handle the important task of product acceptance, It is the 
method used for the latter purpose which I will describe here, 

The traditional method of quality control in the liquor industry 
is the expert system, in which one or a small group of expert tasters 
attempt to handle the job by means of particularized knowledge of 


whiskey and by individual sensory skill. This is a perfect example of 


6 5 


the overconfident approach. The experts mst presume both unerring judg- 


ment and unerring sensory perception. The laboratory methods, in con- 


trast, put taste-testing on a level where both validity and reliability 
of results are markedly improved. 


Control testing implies a standard. The subjective standard used 


by the consumer, and by the expert taster, consisting as it does simply 


of a knowledge of what is good, is too vague, too subject to chance 


variation, to be an accurate guide. 


The solution was to create a physi- 


oal standard embodying the desired flavor. Then testing can be done in 


terms of difference from this standard. Capricious change is thereby 


avoided and one also takes advantage of the fact that difference testing 


can be mich more precise than the testing of absolute values. Originally 


to obtain the standard for a given product the best combination of ingre- 
dients possible, considering time and whiskey stocks available, is 


worked out by a series of taste preference tests using observer groups 


representative of the customer public. This standard is maintained by 


an extension of the same procedure used for acceptance work and any 
variation therefrom is made by prior decision. 
In the production process the whiskey is blended in lots of from 8 
to 10 thousand gallons according to a specific formila worked out by the 
Quality Laboratory which expresses the master standard in terms of the 


partioular whiskeys available. After processing is completed a sample 


of each tank lot is submitted to the laboratory for an answer to the 
question, "Can this lot be distinguished from its standard by sensory 


test?" If the answer is negative, the lot iz accepted; if the answer is 


~ = 


affirmative, the lot is rejected and mist be reworked. When a rejection 
occurs, it indicates that either one or more of the whiskeys or other 


raw materials used were non-standard or else that there has been an 


irregularity in the produotion process. In either case the causes can 


be sought and, if identified, oan be eliminated, 

This system provides for a complete inspection of outgoing product 
at the oompletion of the blending process, since no lot of whiskey is 
sent to the bottling tanks until sampled and approved. The possibility 
of flavor oontamination during the bottling operation is but slight. 
However, the laboratory also makes periodic checks of samples taken from 
the bottling line. 

With this general background established we will go on to specific 
desoription of the test used, including some discussion of controls 
which are necessarye 

Without doubt, the observers, i.e., the persons who do the testing 
and judging, are the most important aspect of the laboratory. Without 
these individuals, without their sensory and perceptual abilities mani- 
fested under conditions of proper oontrol, the system is useless, Their 
seleotion and training cannot be haphazard and their performance must be 
oontinuously subject to review. Ten observers comprise the normal teat- 
ing groupe They are usually women, originally hired to work on the 
bottling lines, who have been assigned to the laboratory on a trial 
basis and have proven their ability. Very likely an observer's sense 
of taste will be more acute than average, but the selection is made upon 
the basis of demonstrated ability to detect differences between whiskies. 
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It should be emphasised that these people are not considered “whiskey 


experts" since their usefulness does not depend upon any kind of umsual 


knowledge or skill. Wot only do they know what they are doing, but we 


know also, and a method is readily available for training replacemat,. 


At present a taste test only is used in acceptance work. At various 


times an odor test has been used in parallel. Howevsr, the two oorre- 
lated highly and, since the odor results were more variable, the latter 


test was dropped in the interests of inspection economy. 


Basically, the test is quite simple. The observers taste 4 oc 


samples of whiskey from ordinary shot glasses. These samples are pre- 
sented in a controlled order and the observer is required to make certain 
statements about them. Special apparatus has been designed to make 
administration of the test speedy yet accurate and the observers have 
been trained to follow procedures in testing which have proven to give 


best results. 


Figure 1, diagrams the procedure of a single test. It shows the 
series of seven taste samples which is given to an observer at a sitting. 
This procedure has been given a name - the “duo-trio" test. 


Prior to the start of the test proper the observer gets as a "warm 


up" semple of 4 oc's of whiskey "C," which is the standard. The "warm-up" 


has this function: The first sample which is tasted may seem quite differ- 


ent from the others merely because of its temporal position and because 


of contrast with the previous neutral condition in the mouth. The "warm 
up” readies the mouth, making conditions more nearly constant for the 


rest of the samples. It is followed by a minimm interval of 20 seconds 


i 
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WARM-UP 


20 sec. 10s 
JtR 


' 


C-somple of standard whiskey X-sample of unknown whiskey 


R- indicates rinsing with water J-indicates judgment on unknown pair 


FIGURE 1. DIACRAM OF THE SAMPLE SERIES WHICH CONSTITUTES 
THE ‘*‘TEST* WHICH IS TAKEN BY AN OBSERVER AT ONE SITTING. 
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during which the observer rinses with distilled water which is always 


available in quantity in the testing booth. In the test proper “C" is 


given first, followed, at 1l0-second intervals, by the two members of 


the "X-c" pair given in an unknown order, Next, there is another 20- 
second interval during which the observer gives his judgment and again 
rinses. Then the process is repeated, i.e., another "C" followed by a 
second unknown pair for judgment. The time required for the entire test 
is about two minutese Test series for a given observer are separated by 
a minimum interval of 15 minutes to give his sensory mechanisms time for 


complete recovery. 


The crux of the whole matter is the response. In this test the 
observer is fully aware of the entire series, i.e., he knows that the 
first sample is a “warm-up” and that in the test proper the first sample 
of a trio is always "C" and is followed by a pair in unknown order. He 
knows that, within the pair, one member will be identical with "c" but 
that the other may differ from it. He mst state which of the two is the 


"different" sample. 


"But," you may object, "suppose he thinks they are 


identical and are both like the standard?" This does arise frequently, 


for "X" will be very close to "C" if the lot from which it was drawn has 


been properly blended. However, the test does not permit a "same" judg- 


ment; the observer mst give his best guess. When we turn to analysis 


of the data we will see why this is necessary. It oan be shown that this 


forcing of a decision is not unrealistic. Rather, it makes for finer 
discriminations. Not infrequently the test as a whole proves that a 


difference exists even though no single obgerver has been certain of 
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his judgment. 
The data are recorded simply as correct or incorrect for each judg- 


ment according to whether the observer has properly identified the "xX" 


or note The complete control test on a given production lot consists of 
ten of these individual test series using ten different observers. Since 
each person gives two judgments there are 20 items for final analysis. 
When results with the 20 judgment test are doubtful, it may be extended 
to 30 judgments by having five observers repeat. 

Controls which are vital if the test is to be effective fall into 
two general classifications. First there are those relating to the test 
samples, "C" and "X," themselves, They mst be identical in every aspect 
under laboratory control to assure that the observers will not get irrele- 
vant cues. 

1. Temperature mst be equal. It has been found that normal 

room temperature is optimum for difference testing. 
Aloohol concentration (proof) must be identical since a differ- 
ence of about 2% of proof is detectable by a trained observer. 

3. There mst be no variation in color. 

4. All samples mst contain the same volume. 

The second class of controls is larger. Their need is dictated by 
common sense, refined by the science of psychology, to assure that the 
observer may work under optimum conditions. Most of them relate to the 
factors of adaptation and fatigue. 

1. As previously explained the "warm-up" sample is given to make 

adaptation uniform 
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The water rinses are employed to slow down the rate of adapta- 
tion to the flavor of whiskey, because, as it proceeds, finer 
differences disappear. 
Also to reterd adaptation, there is a "no-swallowing"™ rule. 
The entire sample is expectorated. When the whiskey reaches 
the deeper regions of the bucoal cavity and the throat it can 


be perceived more fully. However, in that case rinsing is 


relatively ineffective and the net result is reduced sensitivity. 
The samples are reduced to 45° proof, which is half the usual 


whiskey strength. 
flavor and tactual quality as to interfere markedly with taste 


At full strength, whiskey has such a strong 


perception. 


Time intervals between samples mist be controlled with two oon- 


flicting objectives in mind. At least a minimm of recovery time 


mist be allowed but intervals cannot be extended to the point 
where forgetting becomes important, 
The order of presentation within the unknown pairs is determined 
by chancee If it is the same all the time or follows any set 


pattern, constant errors might arise. 


The testing situation mst be kept free from distractions, both 
sensory and ideational, to permit observers to operate at the 
optimum, For example, smoking is not permitted in the laboratory. 
Production personnel are not allowed to discuss results with 


the observers. 


Se 

4. 

| 

| 
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8. Motivation is important. To make the system work the observer 


must consider each unknown pair a challenge. It is made 
explicit that the criterion of job success is the ability to 
make a good percentage of "right" responses. At the end of 


each test the observer is told his results and a continuous 


record is kept of each observer's percentage correct. She 
knows that the penalty for falling singificantly below the 
performance level of the group is return to the bottling line. 
also, a day by day competitive spirit is fostered among the 
observers. 
How do we use these data for control? Remember that all of these 
judgments may be guesses or, on the other hand, they may represent a 
greater or lesser degree of certainty on the observers' part. However, 


all are treated as being of equal walue, Also each judgment is treated 
as a separate datum even though it is a member of a pair and there is 
probably some degree of correlation. 
If blending of the lot under consideration has been normal the "xX" 


will be identical in taste with the standard "C," or quite close to it, 


and most of the judgments will be, actually, guesses. However, over a 

period of time lots will appear which cover quite a range of flavor varia- 
tion. As difference from the standard increases, guessing will become 
less frequent and therefore the percentage of correct responses will be 
higher. 


How high must this percentage go before we can be certain that, 


considering the group of responses as a whole, they no longer oould repre- 


sent guess work? How high before we can sefely say, "Here is a true differ- 


ence3" 
12 
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To answer this question we look at a hypothetical set of data. What 
can we expect to happen when the samples are identical? Here we have a 
situation analogous to the penny-tossing experiment where we record the 
percentage of heads in twenty tosses, for there are but two alternatives, 
and each observer has a 50-50 chance of being right on each judgment, 
Following this analogy further, we know that if we run our taste test on 
identical "X-C" pair and then repeat the test many times over, recording 
the percentage of correct responses obtained each time, these peroentages- 
correct will tend to distribute on the normal ourve with 50% correct as 
the mean. (It has been established by experiment that the results on 
identical samples will so distribute.) Now we inquire as to the frequency 
with which any given percentage-correct will occur. 

We analyze our theoretical distribution in terms of the Standard 


Error, using the formula: 


= 


Standard Error of a percentage 
percentage-correot 

(l-p) or percentage incorrect 
number of cases 


Gp 


q 


Here the true mean is 50%. Therefore, 


e112 or 11.2% 


I: | 
250 
q = 50 
= 
| — 
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The significance of the Gp is this: Upon repetition of the test 


on the identical samples, the result will fall within certain Gp limits 
from the mean peroentage with a frequency which oan be estimated by refer- 
ence to the standard tables of area under the normal curve, Figure 2 is 
a diagram of the situation under discussion. 

The mean perocentage-correct is at 50% Referring to the tables, 
68.2% of all tests will fall within 1Gp each way from 50% -- or between 
61.2% and 38.8% correct judgments <-=- and 95.4% will fall within 2Gp, or 
between 72.4% and 27.6% correct judgments. The expectancy that the result 
of the test will fall beyond any given "Gp" point is represented by the 
area under the curve beyond this point. For example, we could expect 
2.3% of all tests to fall beyond 72.4% correct judgments and the same per- 
cent to fall beyond 27.6% at the lower end of the ourve. 

Table I shows these calculations for all possible percentages using 
twenty judgments, and, for the thirty judgment test, the figures near the 
region which is critical for purposes of this test. Only half of the 
symmetrical curve is tabulated. In these calculations we have assumed 
a normal curve for the data. Since we are dealing with a binomial 
' expansion this assumption is not strictly correct but it is a close 
enough approximation for this particular application. Columns 1, 2, and 
3 show, respectively, total judgments, number correct, and percent correct 
into the Gp distance from 50%. Colum 5 shows the percent of area under 
the curve between 50% and this point and Colum 6, the percent beyond. 
Colum 6 has another meaning since it also gives the chances in one-hun- 


dred of equalling or exceeding the given percentage correct. 
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TABLE I 


RATINGS AND AREAS UNDER THE NORMAL CURVE 
CORRESPONDING TO GIVEN PERCENTAGES CORRECT 


SE Distance Percent area Percent area 
Total Number Percent from 50% between 50% beyond this 
Judgments Correct Correct Gp rating) and this point point 


20 50.9 
65.0 
70.0 
75.0 
80.0 
85.0 
90.0 
95.0 
100.0 


0.00 
15.6 
31.6 
40.3 
46.4 
48.6 
49.7 
49.9 
50.0- 
50.0= 
50.0- 


eee 
O 


60.0 
63.3 
66.7 
70.0 
7323 
76.7 
80.0 


avn n 


36.4 
43.3 
46.4 
48.6 
49.6 
49.8 
50.0=- 


FREQUENCY 


f 
38.8 50.0 612 


PERCENT CORRECT RESPONSES 


FIGURE 2. THEOKETICAL DISTRIBUTION OF PERCENTACES CORRECT 
UPON REPETITION OF THE TASTE-DIFFERENCE TEST ON IDENTICAL 


SAMPLES USING 20 JUDUES. SHADED AREA REPRESENTS PRODUCTION 
RISK WITH THE 2.20 REJECTION POINT. 


l4a 


50.0 
34.4 
18.4 
9.7 
3.6 
1.4 
0.5 
0.1 
Very small 
n 
30 18 13.6 
30 19 6.7 
30 20 3.6 
| 30 21 1.4 
4 30 23 0.2 
30 24 Very small | 
| 34.1% | 341% | 
3 
' 
2.20 
13.6% 13.6% 
| 
-20 


To test for difference we compare any given result on an unknown 
lot with this theoretioal distribution and find out how often the result 
might arise with identical samples, If the latter frequency is low we 
conclude that there ie a true difference. 


The reader may be interested in the significance of percentage- 


correct results below 50%. They usually represent nothing more than 


chance expectancy with identical samples. However, if any percentage is 
lower than chance reasonably would allow, we mst conolude that the test 
was faulty -- either that the observers were making some type of con- 
stant error or else that for some reason they were trying to defeat the 
test in order to approve the sample. In either case the test is repeated 
with additional precauti ons. 

With this analysis behind us we still have the problem of seleoting 
a workable rejection point, one which gives assurance of customer satis- 
faction, yet one which can be met by production without too great expense. 
The breaking point at present in use is 2.2Gp, which point is indicated 
in Figure 2. Any result below this point approves the lot, at or beyond, 
rejects it. Referring to Table I we see that, at this point, there are 
only 1.4 chances in 100 that the lot could be identical with the standard. 
Note, however, that this does not explicitly tell anything about the 
degree of difference between the two, It is a logical inference that 
greater "Gp" ratings reflect larger degrees of difference, but the method 
does not require this to be true. In effect we have said only that it is 
highly improbable that the lot is identical with the standards definitely 


we have not said that the difference is such that we should always expect 
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such a result. In fact a repetition of the test is quite likely to 
result in a lower percentage, although never as low as 50%, 


Following are examples of the system in operation. They oan be 
checked by reference to Table I. 


Example 1: Number of judgments <-- 20, 


Number correct -— 13 

Cp rating -- 1.5 

Disposition =~ The lot is approved. 
Example 2: Number of judgments <- 20. 

Number correct 16 

O p rating 2.7 

Disposition -~ The lot is rejected 


and mst be reblended,. 


Example 3: Let us say that the 20-judgment test showed 
70% correct. TheCp rating would be 1.8. 
This is considered doubtful and the test is 


continued. 

New total = 30. 
Number correct -- 19 

Op rating 1.5 
Disposition Approved 


Example 4: The 20-judgment test shows 75% oorrect — 
OC p rating, 2.2. Again this is borderline 
and the test is extended. 


New total 30. 
Eumber correot -- 21. 
CO p rating “= 2.2 
Disposition . == Rejected and reblended. 


This method does not attempt to "prove" that the production lots are 
the same as the standard. Rather, it implies identity for practical pur- 
poses, through failure to prove that a difference exists. This is the 
customer's assurance of quality. With an approved rating of 1.80p, it 
is still quite possible that there is a true difference, but since, even 


16 


under the optimum laboratory conditions, we cannot prove the fact, we 
know that under field cenditions of liquor consumption, the difference will 
be unnoticeable. 
Replenishment of standards represents a more difficult problem than 


the testing of production lots because here the acoumlation of even 


slight differences, over a period of time, could lead us away from the 
original taste quality. The solution is to extend testing of potential 
standard to five times the usual test-length, 100 judgments, and to accept 


lots only where results are internally consistent snd where the combined 


percent-correct lies close to 50%. Then we do have fair statistical 
assurance of identity. 
Note that production must take the risk of having rejected, on the 
average of 1.4 times in 100, lots which are identical with the standard. 
There is a much greater, but unknown, risk of rejecting lots which differ 
in such slight degree that they would have no adverse effect on sales. 
Of course, in the final analysis limits are set, not on the basis of 
statistical theory alone, but also on the basis of what production oan 


achieve, and at such a point that the cost of reblending, and the incident 


delay, do not become prohibitive. At the outset of the program limits 


were moh wider than at present, but, under the strong motivation pro- 


vided by the system, and aided by the definite information it oould give, 


the production department did such a good job of process control that 
limits could be progressively narrowed. 


We should note that a producer of alcoholic beverages can use this 


system to greater advantage than most food processors because his product 
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is quite stable. If you put a sample of whiskey in a sealed pyrex bottle 


it will retain its full flavor indefinitely. There are no worries abaut 


deterioration or possible contamination. This permits the distiller to 


maintain an actual physical standard over a long period of time. Few 


food products are like this, time is an important factor in their flavor 


quality. 
standard will remain constant in flavor for an extended length of time. 


More often than not there is no assurance that a physical 


In such cases the difference testing procedure exactly as used by Seagram's 
is hardly applicable and we mist seek stable reference points elsewhere. 
The run-of-the-mill solution has been to use a completely subjective 


standard as represented by the memories of a few trained individuals, In 


other words, you rely on persons who simply remember the flavor. How- 
ever, it has been demonstrated many times that this is a make-shift 


arrangement. Its measurements are unreliable and the standard it repre- 


sents is not standard enough to be fully effective. 
Some methodological work is now underway at the Quartermaster Food 


and Container Institute that has a bearing on this problem. I will tell 


you briefly of two phases of our work. However, I wish to emphasize 
that they do not represent finished flavor control systems but are only 
suggestive of the direotions which such work may take, 
The first relates to dried milk, but might be applicable to any pro- 


duct where there is the same difficulty of maintaining a standard. Dried 


whole milk is one of the problem children of the dairy industry. It shows 
high variability in quality, ranging from material fit only for animal 


feed to that which might even complete for the fresh milk market. A good 
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deal of effort has gone into flavor evaluation in attempts to improve 
and control quality, most of it being done with the subjective standard 
and expert panels, This work has assisted process research in slowly 
raising standards, but it has not given effeotive control of flavor 
quality. 

Our experiment started with the reasonable assumption (which has been 
verified by consumer preference tests) that quality in a dried whole milk 
depends upon how closely its flavor approximates that of fresh milk. 
Even though this is generally conceded, such knowledge seems to do little 
good in a test involving subjective evaluation, orobably because the 
sensory difference between the two types of milk is so marked, 

We set up an index of quality called Dilution Number. This index 
we defined as: "The percent of dried whole milk in a mixture of this 
substance with a fresh milk standard suoh that the difference in flavor 
between the mixture and the standard is barely detectable." In other 
words, how much of the dried milk must be put in fresh milk before the 
flavor change is detectable. 

To establish the index for a given sample, we set up a series of 
dilutions, differing by small increments of percentage of dried milk, 
and such that their flavor seemed to be close to the point of a being 
just detectably different from fresh milk. These dilutions were tested 
one efter another for difference against the fresh milk standard using 
the duo-trio test with a total of 20 judgments. The oritical dilution 


was that which gave just 15 ocorreot responses out of 20. Refer to Table 


I and you will note that this is the first point which goes beyond 2Op. 
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A typical set of results is shown in Figure 5, which also shows calcu- 
lation of the Dilution Number when no dilution gave exactly 15 correct 
responsese 

Our experimental work with this system was encoureging, though far 
from adequate for “proving” it. Table II summarizes the results from 
one experimental run. We established DN's for 11 samples of dried milk 


selected so as to represent a range of quality, re-established them later 


to oheck reproducibility, and also submitted them to that ultimate board 


of appeal, a consumer preference test. The reproducibility of the index 
was found to be high, as evidenced by a correlation of +.96 between the 
first and second series. There was « correlation of +.89 between the 
rank orders of the samples by Dilution Mumber and by consumer preference 
ratings, which was equally encouraging. 

A logical next step in verification of this method would be to have 
a manufacturer use it routinely in order to find out how effective it 
would be in a practical situation. Of course, a control laboratory 
would not be able to do as much work on each sample as we did. But here 
is the way the system might be used: By careful test the laboratory would 
establish a DN for the flavor quality the plant could consistently pro- 
duce. Let us say the DN turns out to be 30. They also would establish 
the DN for quality below which they know they would not want to go. 
This might be 25. The following tests would be rum on a representative 


sample of each lot, or, for a continuous process, at fixed time intervals: 
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TABLE II - DILUTION NUMBER AND CONSUMER ACCEPTANCE 
RATINGS OF ELEVEN DRIED MILKS 


First Second Average Consumer 
Code DN* DN DN Acceptance** 
(a) (b) 


A 30.0 32.5 31.2 4.9 
3 27.5 25.0 26.2 4.6 
Cc 25.0 22.5 23.8 5.2 
D 22.0 25.0 23.5 1 Sel 
23.0 22.5 22.8 
F 23.0 20.0 21.5 4.5 
G 19.0 23.0 21.0 3.6 
H 3.0 15.0 14.0 4.4 
I 10.0 14.¢ 12.2 3.2 
J 8.5 8.0 8.2 3.0 
K 0.3 0.5 0.4 2.4 


Product moment correlation between (a) and (b) = +.96 
Rank order correlation between (c) and (d) = 4.89 


* - Dilution Number 
** - By a 9=-point "like-dislike” scale 


19 - 
18 - 
17 - 
16 - 
15 - 


14 
13 - 


Number of Correct Judgments 


+ + + — 
15 20 25 8) 
Percent Reoonstituted Dry Milk 


Figure 3 Method of determining Dilution Number 
208 


| 
| 
difference 
\ 
‘ 
. 
ll | DN 18,3 


Test #1. At DN = SO (Fresh milk vs. a mixture of 30% product 
plus 70% fresh milk.) 


a. Result below 2Gp - approval. Production is in con- 
trol; no more testing necessary. 


be Result 26 Pp or above - non-approval. Test further. 


Test #2. At DN = 25 


a. Result below 20p = approval. However, the product 
ie suspect and the process should be investigated. 


be Result at or beyond 20p = rejection. Product is 
sub-standard and process is out of control. 


The second problem which we worked on at the Quartermaster Food and 
Container Institute is that of establishing a valid measure for acoept- 
ance of a product for which no natural standard exists. Soluble coffee 
is a good example, bouillon another. We are working on both of these 
products at the present time. The use of an arbitrary physical standard 
in such cases might result in great economic waste by rejecting material 
which, though quite different in flavor from the standard, would be 
equally as good in terms of the mass consumer response. In this case the 
testing philosophy moves closer to the final criterion of preference. A 
consumer's reaction will be one of liking (or appreciation of good quality), 
which will lead to acceptance, or else one of dislike (or awareness of 
poor quality), which will lead toward relative non-acceptance. Here, to 
accomplish the necessary task of making a direct estimate of the con- 
sumer reaction, a method is used in which precision is much poorer than : 
in the difference testing previously discussed, but where validity oan be 


easily upheld. It utilizes the quality judgments of a special panel of 
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observers as expressed on a rating scale, 


We use a panel of 20 persons who have been selected on the basis 


of ability to make fine discriminations of differences in the product 


under investigation (which is the same basis of selection as used for the 


Seagram-Calvert observers) and also on the basis of ability to reproduce 
judgments of quality. 
by taste-testing a variety of samples covering the normally expected 


range of quality. 
Figure 4 shows the test form which is used in evaluation of soluble 


These people are familiarized with the product 


coffee, inoluding both instructions and the rating scale. Note that this 
scale calls for a judgment of quality rather than for a “like-dislike" 


response such as would be used in a consumer preference test. Panel 


members are instructed to consider good quality that which they believe 


will have high consumer preference even though this may not agree with 


their personal preferences in all cases. They work independently while 


testing. Values assigned by the 20 people are then averaged to get the 


final rating. 


The panel will generate its own standard for acceptance by evalua- 


ting representative samples of a mumber of brands, or grades, of a pro- 


duct. As many samples are tested as necessary to give a reasonably good 


estimate of the spread of mean scale values which may be expected con- 


sidering everything on the market, and sufficient data are taken to per- 


mit a good estimate of the precision of a single mean value, Table III 
shows a set of typical data from this method and how they would be used 


to set an acceptance standard. 
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TABLE III - EVALUATION OF A SERIES OF SAMPLES OF 
SOLUBLE COFFEE ON THE 9=POINT JUDGNENTAL 
SCALE BY A 20 MENBER PANEL 


Mean Rating Standard Deviation 
Xz, Sample X G2 G3 Sample6 


7.3 7.0 Tel 


6.8 6.7 6.8 a) 1.0 

6.2 6.1 1.0 09 

4.9 522 5.0 1.2 1.1 

4.3 4.8 1.1 1.2 

3.8 4.2 4.1 4.0 1.4 1.2 


Grand mean of all samples - 5.6 Average of sample G's - 1.05 
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36x = 69 


A 6.9 

C 

D 

F 

22a 


COFFEE RATING SCALE 


You will receive one or more samples of coffee, one at a time, 
which you are to judge as to quality. Evaluate each by the phrases 
printed below the scale. Write the sample code at the left and 
indicate your judgment by a check mark on the scale. 


You may swallow the coffee if you choose but treat all samples 
alike. Rinse your mouth thoroughly after each sample, 


This is not a preference test. You are to judge each coffee in 
terms of how acceptable you believe it would be to most consumers. 


4 6 7 8 9 


Extremely Very Poor Below Fair ir Below Good Good Very Excellent 
Poor Poor Above Poor Above Fair Good 


3 4 
T 


3 


FIGURE 4 - RATING SCALE USED IN JUDGMENTAL 
EVALUATION OF SOLUBLE COFFEE 


| 
Sample 
Number 
1 2 po 5 6 I 7 8 9 
l 2 | a 5 6 7 8 9 
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The panel tested each sample of coffee three different times. The 
mean soale value and standard deviation are shown for each test, followed 
by a "Sample X" and "Sample © “ which are in each oase the averages of 
the three values. Sample G's for the six samples are averaged, giving a 
value of 1,05, which is an estimate of the variability which oan normally 


can be expected in the distribution of judgments when this panel rates a 


coffee. 


From this we calculate the standard error of the mean when 20 
judgments are taken: 


= 023 
20 


This is the estimte of the normally expected variability of a mean rating. 


The "Sample Y" values serve as a guide in setting the quality standard 
although just what level should be selected still remains a matter for 


decision. The "grand mean of all samples" is one logical value to use, 
This would represent a decision to accept anything which was not provably 
in the low quality range. This plan would have partioular merit if the 
products whioh were tested to establish the standard gave mean scores which 


were distributed normally. Top quality could be demanded by setting the 


standard at 7,1, which was the best mean score found with the coffees 
tested, 


Still another method that has good justification is this. Note that 


two samples in the series tested here have mean ratings that lie more than 


30y below the "grand mean of all samples." They are obviously of inferior 
quality and might for that reason be disregarded in setting the standard 


value. For example, here one would recalculate the mean for the best 
four samples and get 6.25 as the standard score. 

In any case, after the standard score is set, the point of rejection 
on the basis of a single mean rating by the 20 member panel would be set 


at 0.69 points (36 y) below the standard score, Perhaps it could be set 


at 25Oy below the standard if one wanted to tighten up on control. 

This system is now being put to practical use. Many military speci- 
fications for food items oarry the general requirement that the material 
must be of “acceptable flavore" This is above and beyond any of the physi- 
cal or chemical specifications that are subjeot to the usual tests, Our 
organization has the responsibility of determining, for beverage items, 
whether pre-bid samples are acceptable in flavor. Here is exactly the 
situation which I have described on a theoretical basis. There oan be no 
single physical standard, sinoe items may vary considerably in flavor yet 
be equally acceptable. Again, we would not be justified in setting a 
standard any higher than the "grand mean of all samples" considering the 
usual wording of such specifications. If we reject any semple we must be 
able to prove that it is definitely below standard, or, as we would have 
to say, "Non-acceptable." 

We can not say yet how practical the method will be as we are still 
in the process of selecting panels and setting up standards, but work is 
underway on synthetic beverage bases, soluble coffee, and bouillon. Once 
we have selected our panels and have calibrated them on the partioular 
products, then we mast wait and see how effective a control will result. 
One thing we know, however, is that for all its possibilities of inaccuracy, 
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this system will be far better than the semi-arbitrary "expert" type of 


sample screening it supplants. Such a rational, explainable method not 
only will give us some measure of assurance in our decisions, it will also 
help the manufacturer for he will be able to find out what the word "acoept- 
able" in the specification means. 

My purpose in this disoussion has been two-fold. First, to tell you 
something of the difficult problem of oontrol of flavor quality; and, 
second, to indicate that at least some people are approaching that problem 
with the same soientifio rigor that is doctrine in the field of Statistical 
Quality Control. I will readily ooncede that it is unlikely that flavor 
control will ever attain the precision that is possible where control 
characteristics oan be measured more objectively. However, great improve- 
ment can be made over the methods that are common practice in industry 


todays 


| 
‘ 
: 
6 25 


NO. 7 
Price 25¢ 


BALANCING AND RANDOMIZING IN EXPERIMENTS 


by 


Warren R. Purcell 
Manager of Quality Control 
Lighting Division 
Sylvania Electric Products, Inc. 
Salem, Massachusetts 


for presentation at the 


FOURTH NATIONAL CONVENTION 
and 
FIFTH MIDWEST CONFERENCE 


of the 


AMERICAN SOCIETY for 


June 1 and 2, 1950 
Milwaukee Auditorium, Milwaukee, Wisconsin 


REPRINT: Subject to revision. Permission to publish this paper, 
in full or in part, after its presentation and with credit to the 
author and the Society may be obtained upon request. The Society 


is not responsible for statements or opinions advanced in papers 
or discussions at its meetings. 


FIFTH MIDWEST QUALITY CONTROL CONFERENCE 
P. O. Box 1204, Milwaukee 1, Wisconsin 


| 

<AN 5 

3 

» = 
3 


BALANCING AND RANDOMIZING IN EXPERIMENTS 
By 
Warren R. Purcell 
Manager of Quality Control, Lighting Division 
Sylvania Electric Products Ince 
Fellow, American Society for Quality Control 


Boston Society for Quality Control 


Introduction 


The Modern Quality Control Engineer is a vital cog in the wheels of today's 


industry. 


His services have become all-important in the ever-present 
problem of identifying, eliminating, and preventing the causes of excessive 


variability in manufacturing. By the judicious and skillful use of various 


statistical and graphical tools, such as histograms and control charts, he 
has been able to help substantially in identifying causes which elude 


identification by old-fashioned methods. 


Occasionally, however, he has encountered difficult problems of cause 


identification, especially in the more complex and technical processes. 


The solutions of such problems often require full-scale engineering in- 


vestigations. In the early days of Modern Quality Control, such investi- 


gations were considered a separate problem to be solved by Material and 


Process Engineeringe It is now realized that the Quality Control Engineer 


has at his command additional statistical and graphical tools, such as 
variance analysis and correlation, to assist materially in the interpre- 


tation of the data yielded by engineering investigations. 


‘ 


A necessary requirement for the effective use of these specialized tools is 
a "way of thinking" in terms of variability and probability. Herein lies the 
real value of the Modern Quality Control Engineer. Because of his way of 
thinking, his services are valuable not only in the interpretation of data 
from experiments, but also in the planning and executing of the experiments 
themselves. Tims his usefulness is now extending beyond the problems of 
day-to-day process control, important as they are, to the fields of research, 
development, and design. Extraneous causes of variability not only disturb 
established processes, but also enter into experiments and often make the 
results tell costly lies. The fallibility of human beings and the perver- 
sity of inanimate objects are constantly rearing their heads to plague the 
investigator. The Quality Control Engineer is, in many cases, the one person 


in the organization fully trained to cope with these intruders. 


Fallaczous Procedures 


One of the most troublesome of the perversities of inanimate objects is the 
tendency of true causes to disguise themselves and to try to appear as some- 
thing else. Consider a few illustrations. 


Not long ago a packing engineer devised a new package for a certain product. 


Adoption of the new package would have saved several thousands of dollars 
annually. Before final adoption a transportation test was run to indicate 
whether the proposed packing would properly protect the product. In the test 
1000 inspected units were shipped on a round trip in the proposed packing 
and 1000 inspected units in the standard packing. Upon their return they 
were again inspectede In the standard packing one unit was found to be 
defective; in the proposed packing 10 units were defective. The decision to 
forego the contemplated saving had almost been reached when it was pointed 


out that the defects could have been due to weakness in the product rather 
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than to inadequate packing protectione Further investigation disclosed that 
this was actually the case, and that the proposed packing could, in fact, be 


adopted. The lie told by this experiment could have cost thousands of dollars. 


At another time an engineer ran a test to determine the relative strengths of 
lamp filaments made from two different types of tungsten wiree To test the 

strength, 20 lamps were mamfactured using one type of wire, and 20 using the 
other ‘typee The lamps were then put through a standard "bump" test to deter= 


mine the relative percentages of filament breakage which would result. In 


this test the glass bulb occasionally shatters and spoils the test of one of 


the lamps. But in this particular test, a strange result happened. Of the 

20 lamps made with one type of wire, none of the bulbs shattered; of the 20 
lamps made with the other type, 10 of the bulbs shattered. The conclusion 
indicated was that one type of wire caused brittle bulbs. The lie told by 
this experiment was so ridiculous that it fooled no onee There was, there- 
fore, little lost in this case, unless we count the time, effort, and material 
expended, plus the loss of faith by the engineer in experimental results in 


generale 


Another case is of a somewhat more familiar typee The life of a particular 
size of lamp was running shorter than its design called for. A suspect was 

the speed of the sealing machine. An experiment was therefore run in which 
lamps were manufactured both at the standard speed and at a slower speed. 

life tests were completed, and the lamps made at the slower speed had substan- 
tially longer life. Even though it meant a loss in production, the anticipated 
increase in life was important, and the machine speed was therefore reducede 
Unfortunately, the life remained just as it had been at standard speed; 

nothing was gained when the speed was reduced in actual production. The lie 
told by this experiment cost about ten thousand dollars in lost production, 


and the problem of short life remained unsolved. 


These are actual cases from our files. Others could be cited. Such cases 
indicated to us the existence of serious fallacies in our procedures. If 
such obviously erroneous conclusions could be drawn, in how many cases were 
less obvious, but no less costly, wrong decisions being made? Every decision 
which is based on experimental evidence is exposed to this same hazard. What 
can be done about it? 

Quantity of Evidence 


One approach is to require that every important experimental decision be 


based on a large quantity of evidence. "Can you repeat the experiment (sev- 


eral times, perhaps) and keep getting the same result?" is the question often 
asked by one who uses this approach. Perhaps the suggestion is made to "try 

it in actual production and see what happens". But these procedures can be 
costly. If they produce positive evidence, they may be well worth while, 

But often the results became conflicting and inconclusive. A relationship 
that is established by today's experiment does not hold tomorrow. Engineer 
Smith finds that an increase in furnace temperature at a certain process stage 
will yield a more wear-resistant surface, but Engineer Jones finds that such 

a temperature increase is without effect on surface condition, or even makes 
it less wear-resistant. Such conflicting evidence is caused by the same kind 
of faulty procedures that make the individual experiment tell such costly lies. 
Causes not only try to disguise themselves, but they sometimes hide completely, 
and at other times come into full view, all with a bewildering sequence of un- 


predictability. 


Control of Conditions 


A more satisfactory approach is to seek quality of evidence rather than mere 


quantity. Traditionally, quality of evidence has been sought by controlled 
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conditions, in order to reduce the effects of extraneous causes. Controlled 
conditions, however, are elusive. No one can ever be sure just how well- 
controlled they are for any one experiment. In many cases control of con 
ditions begs the question, for control is often the very objective of the 
experiment. In other cases control of conditions is properly challenged on 
the ground that an artificial "laboratory" result will not necessarily hold 


true in the everyday "factory". 


In field tests, such as the transportation test cited, conditions are practi- 
cally uncontrolled. This is as it should be. Careful execution of the test 
to avoid severe handling, for instance, would itself vitiate the test, for 
the objective is to study conditions as nearly as possible as they will be 
expected in practice. In factory tests, conditions are usually scmewhat con- 
trolled, for this is, or should be, the normal factory situation. In labora- 


tory tests, conditions are usually as closely controlled as the existing 


facilities and abilities will permite 


These various degrees of control affect the well-knowm "experimental error". 
Other things being equal, closer control means less experimental error. This 
has long been recognized. It has also been recognized that experimental 
error tends to obscure the result. What has often been overlooked is that 
obscuring of a result is only a small part of the havoc which can be wrought 
by extraneous causes. The results of the transportation test were not 
obscure; they told a clear lie. The bump test result was not obscure; it 
was ridiculous. The result of the machine speed test was not obscure; it 
told a ten-thousand dollar lie. 


The New Techniques 


Obscurity can be reduced by control, but often at the price of using a narrow 


and misleading set of artificial conditions. Even close control, however, does 


not remove the possibility that clear lies will be told by the result. These 


lies can be adequately prevented only by one or both of two additional tech- 


niquese One of these is "balancing"; the other is "randomizing". By means of 


balancing, the possible effects of selected extraneous causes are separated 


from the effects of the cause or causes being investigated. By means of 


randomizing, the possible effects of all other extraneous causes are neutralized. 


Furthermore, these techniques accomplish these ends whatever the degree of 


controle 


Transportation Test 


Consider the transportation test. This test had been executed by shipping all 
of the 2000 units in one truck. All of the 1000 units shipped in standard 


packing, however, had been manufactured on one day; all of the 1000 units 


shipped in the proposed packing had been manufactured on a different day. The 
conclusion that the proposed packing was inferior was therefore without sound 
basis. The effects of any possible differences between manufacturing dates 
had been completely confounded with any difference in the effects of the two 


types of packinge 


Questioning revealed also that no thought had been given to randomizing the 
order of loading the packages into the truck, handling them at duetinetion, 
etc. Thus it was not unlikely that severity of treatment, as well as a dif- 
ference between manufacturing dates, could also have been badly confounded 


with any effects of the different types of packing. 
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It is clear that, if all 2000 units had been manufactured on one date instead 


of two, there would have been no confounding of effects of type of packing 


with effects of differences between manufacturing dates. There would, however, 


have been a narrow base for the experiment. If a difference between manufac- 


turing dates had existed, then conclusions valid for one date would not neces- 


sarily have been valid for another date. In any case, an experiment based on 


one date is incapable of shedding any light on differences between datese 


Confounding of between-date effects could have bee avoided and the two dates 


still included in this instance by an extremely simple plan of balancing. 


In each type of packing, there were 100 units to the package. There were, 


therefore, 10 packages of standard packing and 10 packages of proposede A 


simple balanced set for the experiment would therefore consist of: 


5 packages (500 uniis) standard packing, manufacturing date I 
5 packages (500 units) standard packing, manufacturing date II 


5 packages (500 units) proposed packing, manufacturing date I 


5 packages (500 units) proposed packing, mamfacturing date II 


Such a set would have balanced the effects of manufacturing dates and types of 
packing, and these two effects could have been separately analyzed, each with- 


out any confounding due to the inclusion of the other. 


There might, however, have been other effects within the transportation trip 
and within each of the mamufacturing dates which could also have confounded 


the results. 


Suppose, for instance, that all 10 packages of date I had been placed at the 
front of the truck and all 10 packages of date II at the rear; or, probably 


worse, suppose that all 10 proposed packages had been in a lower layer of the 
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loaded truck and all 10 standard packages in an upper layer. Im either case 
any differences in transportation severity due to these locations would have 


been confounded with the effects which were being investigated. 


It is clear that, if the packages had been loaded on the truck in a truly 
random order, the placing of all of the proposed packages in a lower layer and 
all of the standard packages in an upper layer would have been extremely un- 
likely. So would any other loading arrangement which could cause a severe corn 
founding of the results, such as the placing of all proposed at the front and 
all standard at the rear; always stacking proposed on proposed, standard on 
standard; placing a large number of one type along a diagonal of the load, etc. 
It is impossible to name all the conceivable reasons for bad confounding, but 


they are all removed from likelihood by the simple process of true randomizing. 


Again, suppose that manufacturing conditions had changed appreciably during a 
day; eege, that product made in the latter part of the day was less shock- 
resistant than that made earlier in the daye Suppose, furthermore, that the 
1000 units selected for standard packing had been made in the early part of 
each of the two mamfacturing dates, while the 1000 units selected for pro=- 
posed packing had been made in the latter part of each of the two manufactur- 
ing datese In such a case also the experiment would have been confounded, and 
in the analysis this difference, really due to manufacturing conditions, would 


have appeared as evidence that the proposed packing was inferior. 


Again it is clear that, if the units for the proposed and standard packing had 


been selected from the production flow by a truly random method, it would have 


been extremely unlikely that the assignment of units to type of packing would 
systematically follow fluctuations in manufacturing conditions to an extent 


which would cause severe confoundinge 


The Fallacy 


of Alternating 


Some experimenters attempt to circumvent the problem of randomizing by alter- 


nating, on the assumption that such a procedure accomplishes the same thing. 


Reflection, however, indicates that this is not S0Oe 


Consider, first, the truck loading. 


Suppose that the load of 20 packages were 


placed in the truck in two layers, each consisting of two rows of 5 packages. 


A normal way to pack such a load might be to place packages in some order such 


as indicated in the attached sketch. 


If, in this sketch, the observer is look- 


ing toward the front of the truck, the packages would be placed in this order: 


First package: 

Second package: front, 
Third package: front, 
Fourth package: front, 
Fifth package: 


Eighth package: 
Ninth package: 


Twentieth package: 


front, left, lower 
right, lower 
left, upper 
right, upper 


second from front, left, lower 


second from front, right, upper 
third from front, left, lower 


fifth from front, right, upper 
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POSSIBLE METHODICAL 


LOADING OF TRUCK 


If, now, the loading is done by "alternating" in the following order: 


standard date I 
standard date II 
proposed date I 
proposed date II 


standard date I 


etce 


all of the standard packing will find its way to the lower layer, and all of 
the proposed to the upper; all of date I will be at the left, and all of date 


II at the right. 


Consider, second, the selection of units for the proposed and standard packing. 
Suppose that units are selected as they come along a conveyor fed by a machine. 
Suppose further that alternating is done by assigning the first unit to pro- 


posed, second to standard, third to proposed, etc. 


Such a machine might well have an even number of heads. Suppose that there are 
10 heads, and that head number 7 is making a weaker product than the reste The 
units made on head mumber 7 will all find their way into one type of packing. 

Which one, of course, will depend on when the selection starts. In either case, 
however, there would be severe confounding of this manufacturing fault with the 


effect of type of packing. 


Planned Randomizing 


Usually randomizing cannot be accomplished by the simple expedient of issuing 
instructions to "load in random order" or the like. If, for instance, in the 
truck loading case discussed, the standard and proposed packages were different 
in appearance or had been externally identified for the purpose of the test, 
the shippers would have been quite likely to load first all of one group, then 
all of another, etc.; for the simple reason that they would want to keep track 
of what they were doinge 


It is far safer to do intentional and planned randomizing. In this case, for 
instance, a card drawing scheme would be applicable. This could be carried 


out by assigning to each of the fovr groups one of the four suits of a standard 
deck of cards, for instance: 


standard, date I 
standard, date II 
proposed, date I hearts 
proposed, date II spades 


If, now, 5 cards of each of the four suits are selected, these 20 cards could 
represent the 20 packages. After thorough shuffling, these cards could then 


be drawn in random order, and numbers assigned to the packages in the order in 


which the cards are drawn. During the card drawing the assigned numbers (from 


1 to 20) would be clearly marked on the outside of the packages and the order 
of drawing recorded and preserved. for subsequent identification. Then the 
shippers and all others who handle the product would be instructed to handle 


"in numerical order", 


The Validity of Conclusions 


After the test had been completed, the results could then be conveniently tab- 
ulated by recording the mumber of defective units in each of the 20 packages, 


tabu...<d by groupse This is indicated in the results of an actual test, very 


similar in nature, and carried out by a similar plan of balancing and randomiz- 


ing. The final data were tabulated thus: 


Date I Date IL Total 
0 
2 
Standard 1 0 8 
1 1 
3 0 
2 1 
0 0 
Proposed 2 10 
1 1 
2 1 
Total Wy 4 18 


In this case it was nearly evident from the raw data that a real difference 
between mamfacturing dates had been show, while no real difference had been 
established between types of packinge 


A statistician, called in at the end of the experiment to analyze these data, 
would have reached substantially the same conclusions, except that he would 


have been likely to express them more cautiously. 


He might have said, for 
instance, that the indicated difference (8 and 10) between types of packing 
could easily have occurred by chance. He would also have said that the in- 


dicated difference (1) and h) between manufacturing dates is greater than 


could reasonably be due to chance acting alone. It would also have been 
apparent to him tiat this indicated difference could not have been due to 
type of packing. 


For type of packing was a selected cause which had been 


separated by balancing. Even if a difference in effect between types of pack- 


ing had existed, it could not have vitiated the comparison between manufactur=- 


ing dates, because there were the same number of units of each manufacturing 


date in each type of packing. 


The statistician's cautious statement that the "indicated" difference between 
mamfacturing dates appears to be real; iee., must in part at least be due to 
causes which did not follow the laws of chance, would have been about as far 
as he could goe He could not have stated that this difference was due to 
manufacturing conditions; iee., existed in the product before the transpor— 
tation test. For the cause “other than chance" which was responsible for the 
observea diiference might, as far as he knows, have been due to peculiarities 


in the teste 


But the engineer who conducted the test knew that this could not be so. He 
knew that his planned randomizing had made every set of causes, except two, 
follow the laws of chance. Those two are the types of packing and the manu- 
facturing datese Since the effect of type of packing had been separated by 
balancing, there remained only one set of causes which could reasonably have 
had an effect “other than chance". That cause was the difference between 
manufacturing dates; i.e., it had existed in the product before the test was 


made, and was due solely to the conditions of manufacturinge 


Note that balancing works both ways. The experiment was planned to study the 
comparative effects of the two types of packing; the between-date difference 
was simply a possible extraneous effect which had been balanced to avoid con= 
founding the main effect. As it turned out, the "extraneous" effect was the 
real onee This real effect was, in turn, kept free of being confounded by any 


difference which might have existed between the effects of the types of 


packinge This would not have been so if the mumber of units of each date's 


manufacture in each of the two types of packing had been unequale 


In the original experiment, in which the two effects were confounded, and from 
which erroneous conclusions had thus been draw, easy assumotions had been made. 
Py implication at least, it had been assumed that any differences in strength 
of product due to time-to-time changes in manufacturing conditions, and any 
effects due to differences in severity of creatment, were small enough to be 
disregarded. But those assumptions were opinions. After the experiment had 
been completed, the conclusions drawn had depended on those opinions; the 


facts had not spoken for themselvese 


Filament Test 


Consider the test of filament strength and its strange result of 10 shattered 
bulbs with one type of tungsten wire and none with the other. The tester 
suggested that a chip of glass must have become embedded in the bump pad 
during the course of the test. The fact that this explanation was offered 
was itself evidence that all of the lamps of one group had been tested first, 
then all of the lanps of the other groupe Perhaps, therefore, it was sheer 
good fortune that the bulbs did shatter. For if a small chip of embedded 
glass can have such a marked effect on bulb breakage, certainly there are 
countless other possible time-to-time variables in test equirment and pro= 
cedures which might have affected the test of the filaments even if the bulbs 
had not brokene Such variables would have vitiated the results anyway, and 
comparisons between types of tungsten wire, the very purpose of the test, 


would not have been valide 


Since these time-to-time variables are entirely unknowm and cannot be enumer- 


ated, their effects cannot be balanced. They can, however, be neutralized by 


lilies 
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randomizinge In this test randomizing would have been extremely simple. If 


the tester had tested one lamp at a time, and determined by, say, the drawing 


of cards, from which group the next lamp should be selected, the test itself 


would have been completely randomized. 


In this case, however, randomizing of the test would not of itself have pro- 
vided a sound basis for valid conclusions. The test was atest of wire, and 
the wire had been through many processes before it had become filaments mounted 
inside incandescent lampse To enumerate only a few, there were coiling, an- 
nealing, treating, sintering, mounting, sealing, exhausting, and flashing. 
Since any one of these processes could have affected filament strength, its 
effects could become severely confounded with differences in the original wire 


itself unless they also were neutralized. 


Consider, for instance, the mounting processe Several sources of variability 
can be named, countless others must exist. Among the namables are the tight- 
ness of clamping of lead wires to filaments, the spacing of the supports, the 
tension of the filament, etc. Suppose that during the mounting of the test 
filaments the tightness of the clamping had been progressively increasing. If, 
then, all 20 filaments of one group were mounted first, and all 20 filaments 
of the other group afterwards, any effect of clamping tightness would vitiate 
any conclusions about type of wire. If, however, the order of feeding fila- 
ments to the mount machine had been randomized, any effect due to clamping 
variables would have been neutralized and would not have vitiated the con= 


clusionse 


Before the experiment had been started, or during its execution, it would have 
been easy to assume that the effects of clamping tightness were either non- 


existent or small enough to be disregarded. But such an assumption would have 
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been an opinion. 


After the experiment had been completed, the conclusions 
draym would then have depended on that opinion; the facts would not have 


spoken for themselvese 


Once the specimens of wire, the difference between which was to be tested, 


had been selected, 


they should then have been sent through every subsequent 


process, including the test itself, in random order. It is clear, however, 


that the mere feeding of units to a machine in random order will randomize 
every step of the process which is performed by that machine; and the intro= 
duction of the units to any series of processes in random order will randomize 


every one of the processes, provided the units are not sorted into order be- 


tween processeSe 


Machine Speed Test 


Consider next the machine sneed teste It was apparent to one and all that 


"something else", other than maclrine speed, must have entered into the experi- 


ment to give the definite evidence of longer life at the slower speed. Many 


are of the opinion that those "something elses" are simply the perversities of 
nature to which experimenters must become resigned unless they are willing to 
repeat their experiments several times and make no decisions unless consistent 


results can be showne 


To those who have formed the habit of thinking in terms of variability, however, 


such a result was not surprising. Every part of every process is exposed to 


many sets of causes of variability at all times, and it would be fantastic to 
expect all of the rest to behave themselves while a selected one is variede 


The experiment was therefore repeated, this time using the randomizing tech= 


riquese 


The first step was to randomize the mounts to be fed to the sealing machine 
whose speed was to be variede The mounts were the product of the last process 
performed before sealinge The randomizing of mounts, therefore, randomized 
the effects of every process which preceded sealing, and effects due to seal- 
ing would thus be unlikely to be severely confounded with effects due to any 


process which preceded ite 


The random order assigned to the mounts was then maintained through every proc- 


ess subsequent to sealing, including the test itself. Thus all sets of causes, 
both before and after sealing, affected the lamps in random fashion, and the 
only causes which could show a real effect in the results would be those due 


to machine speed itself. 


Analysis of the final results showed that machine speed was without measurable 
effect on life. Further ferreting finally disclosed the real cause in an en= 
tirely different part of the processe There is every reason to believe that, 
if adequate precautions had been taken to neutralize extraneous causes in the 
original experiment, erroneous evidence of an effect due to machine speed would 
have been lackinge For in that first experiment the easy assumption had been 
made that the effects of all other extraneous causes were either non-existent 
or small enough to be disregarded. But such an assumption was an opinion. 
After the experiment had been completed, the conclusions drawn had depended 


on that opinion; the facts had not spoken for themselves. 


Special Designs of Experiments 


Much has been written and said about factorial experiments and their value in 
the study of several sets of causes simultaneously. By means of such balanced 
designs of experiments it is possible to increase both the breadth of the base 


of an experiment and its efficiency in preventing duplication of experimental 


efforte This experimental efficiency is even more spectacular in the designs 


known as Latin Squares and Graeco-Latin Squares. 


Experience indicates, however, that too little emphasis has been placed on the 
importance of randomizing the effects of extraneous causes which are always 


present, even in these ingeniously balanced arrangementse 


Consider a comprehensive experiment being conducted at the present time to 
study the effects of various cathode design parameters on the life of fluores- 
cent lampse The cathode is a tungsten filament of the "coiled coil" typee It 
is manufactured by winding tungsten wire on molybdenum mandrel wire, then wind= 
ing this combination again on a steel mandrel pin which is automatically with- 


drawn when the filament is cut to lengthe The molybdenum mandrel is then ree 


moved by acid treatment. The filaments are mamfactured in "schedules" of 


several hundred or more at a timee 


The five parameters whose effects are being studied are: 
Tungsten wire diameter 
Molybdenum mandrel diameter (Primary mandrel) 
Turns per inch wound on molybdenum mandrel (Primary TP Ia) 
Steel mandrel pin diameter (Secondary mandrel) 


Turns per inch wound on steel mandrel pin (Secondary T.P.I.) 


By selecting five different values of each of the five different parameters, a 
5 x 5 Graeco-Latin Square was laid out to study these five parameters as thor- 


oughly as could be done with a test on only 25 lamps. 


After the plan of the experiment had been arranged, an order was transmitted 
to the Filament Department and attached was a list of the 25 different designs 


of filaments for the experiment. The list was made out as shown below, 
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where the five values of tungsten wire diameter are represented by A, B, C, D, 
E3; the five values of primary mandrel diameter by I, II, III, IV, V; the five 
values of primary T.P.I. by 1, 2, 3, lh, 53 the five values of secondary mandrel 


diameter by a, b, c, d, e; and the five values of secondary T.P.1. by v, w, x, 


Wire Primary Primary Secondary Secondary 
Diame Mandrel Diame le Mandrel Diame TePe le 


I 
II 
Itt 


SANK 


% 


A 
A 
A 
A 
A 
B 
B 
B 
B 
B 
C 
C 
C 
C 
C 
D 
D 
D 
D 
D 
E 
E 
E 
E 
E 


aN 


The combinations of designs in this list met the requirements for a Graeco= 
Latin Square, in that each value of each parameter was associated with each 


value of each of the other parameters once and only oncee 


Made out in this order, however, the list would have implied that the schedules 


of experimental filaments should be manufactured in the listed order. If this 


Ze 
Design 
Noe 
3 
IV 
5 
6 I 
7 II 
8 III 
9 IV 
10 Vv 
I 
12 II ; 
13 III 
14 Iv 
15 
i 16 I 
17 II 
18 III 
19 IV 
20 Vv 
21 I 
22 II 
23 III 
2h IV 
25 


had been done, there would have been serious faults in the procedure. With 
respect to wire diameter, there would have been the fallacy of manufacturing 
all schedules of filaments with wire diameter A, then all with wire diameter 
B, etc. In this way any possible time-to-time trends due to extraneous causes 
would have been severely confounded with the effects of wire diameter. With 
respect to primary mandrel diameter, and to some extent with respect to the 
remaining three parameters, there would have been the fallacy of alternating. 
In this way, any possible cyclical variations, keeping in step or nearly so 


with the alternations, would have caused severe confoundinge 


The order of the list was therefore permitted, by card drawing, and the re- 


vised list as sent to the Filament Department was as shown below. 


Wire Primary Primary Secondary Secondary 
Diam. Mandrel Diam Mandrel Diam. T.Pele 


C 


00 


1 
2 
3 
5 
6 
7 
8 
9 
10 
n 
12 
13 
15 


om: 


a4 


—Noe 
II 
21 
22 
23 
2h 
25 
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This included the same designs as the list which had been laid out in order, 
but the suggested procedure had been completely randomized. In addition to 
revising the list, specific instructions were issued directing that the 
schedules of filaments be mamfactured in the order shown, and the job was 
followed closely to make sure that the listed order was followed to the letter 
at every stage of the filament manufacturing processe 


It was fortunate that this was done, for the following of random order often 
adds to the work involved in carrying out certain parts of a process, and 
there is a temptation to sort, in order to expedite the job. This case was 


no exception, 


For instance, between primary and secondary coiling, the tungsten-molybdemm 
combination is annealed in a hydrogen furnacee The specified temperature of 
the furnace is different for different values of primary mandrel diameter. 
Furthermore, it takes time to change the furnace temperature. In order to 
expedite the experimental procedure at this point, the foreman proposed that 
he use five different furnaces, setting one at the temperature for diameter I, 
a second at the temperature for diameter II, etce, and then run all diameter I 
through the first furnace, all diameter II through the second, etc. The engi- 


neer, of course, could not allow this procedure. The foreman than asked if, 


using a single furnace throughout, he could first set at the temperature for 
diameter I and anneal all five schedules of diameter I, then set at the tem- 
perature for diameter II and anneal all five schedules of diameter II, etc. 


Again the engineer refused, and insisted that the listed order be foliowed. 


This insistence was absolutely necessary. There can be differences, other 
than temperature, between furnaces and within one furnace fran time-to-time. 


Among the possibilities are atmospheres, temperature gradients, etc. The 


effects of such variables belong in the experimental error, and mst not be- 
come associated with a parameter being investigated. Randamizing keeps them 
in the experimental error. The procedures proposed by the foreman would have 
made them become associated with primary mandrel diameter. 


The objection might be made here that, since different primary mandrel diam- 
eters require different annealing temperatures, this temperature effect is 
confounded with primary mandrel diameter anyway. Reflection indicates, how 
ever, that specified changes in annealing temperature are not extraneous 
causes, but are necessarily associated with changes in mandrel diameter, and 
decisions between mandrel diameters necessarily carry with them decisions be- 
tween corresponding annealing temperaturese The purpose of the random pro- 
cedure is to prevent confounding by neutralizing the effects of unknom and 
uncontrolled causes which otherwise might become associated with the para- 


meters during the experiment, but would not necessarily be so associated at 


other timese 


The lamps, now on test, will not reach the ends of their lives until well 

over a year from now. During the experiment, it would have been easy to as— 
sume that these extraneous between-furnace and time-to-time effects were 
either non-existent or small enough to be disregardede But such an assumption 
would have been an opinion. When the data have been assembled, the conclu- 
sions then drawn would depend on that opinion; the facts would not have spoken 
for themselvese Erroneous conclusions, after many months of experiment and 
test, would be a high price to pay for a few hours saved during experimental 
filament processing by work simplification which "derandomizes" the procedures 


Balancing Vse Randomizing 


Frequently there is a clear choice between balancing and randomizing a partic=- 
ular source of variability. Consider, for instance, a recent experiment on 


incandescent lampse 


The objective was to study the effect of varying degrees of filament distortion 
on the life of lamps, and also how this varied between two different types of 
mount construction. Two lamps of construction I and two lamps of construction 
II were selected for each of five different degrees of filament distortion, 
making 20 lamps in all, 


At the life test laboratory there were two test racks, each having 10 sockets. 
Any difference which might exist between racks was therefore a source of vari- 
ability. Such a source of variability would be badly confounded with type of 

lamp construction if all 10 lamps of construction I were tested on rack A and 

all 10 lamps of construction II were tested on rack Be. It would be better to 

neutralize this possible effect by assigning lamps to racks at random. Would 

it be better still to balance this possible effect by assigning one lamp of 


each construction-distortion to each rack? 


In the actual test, the latter course was chosen; i.e., the possible effect 


of the racks was balanced. The final life data (expressed as a percentage 


of rated life) were as shovm below. 


Construction I Construction II 
Rack A Rack B Rack A Rack B 


1 116 
Degree of 2 9h 83 106 13h 10h 
Filament 3 81 93 135 98 102 
Distortion 109 105 129 
0 


Averages: Rack At 107 Construction I: 96 105 


Rack Bs 103 Construction II: 11) 


Analysis of these data indicated that the difference between constructions is 
highly significant, the differences due to distortion are significant, but that 


there was no measurable difference between the racks. 


If, however, there had been a real difference between the racks, the data might 


well have appeared thus? 


Construction II 
Rack A Rack B 


Construction I 
Rack A Rack B 


1 124 107 137 105 18 
Degree of 2 102 BS ny 126 10) 
Filament 3 89 85 143 90 102 
Distortion h 126 101 113 121 5 
a 88 65 110 79 85 


Averages: Rack As" 5 Construction I: 96 105 


Rack Bs 95 Construction IT: 11) 


Analysis in this case would indicate a real difference between racks, and 
would also indicate, as before, a high significance in the difference between 


constructions, and significant differences due to distortion. Thus, even with 


a real difference between racks, the value of the balanced experiment would be 
unimpairede 


Suppose, on the other hand, that with this real difference between racks exist—- 


ing, the test had been randomized instead of balanced. If this had been done, 


each lamp could have been placed on either rack, and the life data might well 


have appeared thus: 


Construction I Construction II Average 


127 105 
10 117 


Degree 


102 126 
of 
89 123 
. 3 10 2 
Filament 
101 121 
h no 
Distortion 93 


65 


Averages 100 110 105 


Analysis in this case would indicate that the difference between constructions 
and the differences due to distortion, both shown to be significant in the bal- 
anced experiment, would appear to ba within reasonable limits of chance and 
therefore not considered significant at all. 


The reason for this is the loss of sensitivity due to randomizing the racks. 
By randomizing racks, an appreciable source of variability would have been 
thrown into the experimental error, and this larger experimental error would 
have made the real effects obscuree In the experiment in which the effect of 
the racks was balanced, the difference between racks was stripped out of the 


experimental error, and this made the experiment more sensitive. 


It is apparent that nothing is lost in any case by balancing, whether the po- 
tential cause has a real effect or not, but that sensitivity can be lost if a 


real cause is randomizede When a clear choice is available, it is better to 
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balance a potential cause which could have been randomized than to randomize 
a real cause which should have been balancede 


The Problems of Application 


The applications of these techniques of balancing and randamizing present to 
the Quality Control Engineer a most interesting and challenging opportunity. 


While the principles are universal, each actual case has its own peculiar . 
physical problems which tax the ingemity. 


The greatest of the problems, how 


ever, are the human ones. The perversity of inanimate objects can be neutral- 


ized, but the fallibility of human beings will present difficulties for many 


yearse One of these is the tendency persons have toward order. This is nat- 
ural, for most tasks are best accomplished when carried out in an orderly 


mannere In investigations of the nature we have been discussing, however, 


order can be carried too far. This will be so when operators and testers 


"derandomize" the procedures by sorting. It has been found necessary to take 


special precautions to prevent this. Careful instructions and close follow 
up are essential. 


A more difficult problem in the human relations involved is the convincing of 
various persons that careful randomizing is actually a necessary prerequisite 
to valid conclusions. Generally the more familiar a person is with a process, 


the more difficult it is to convince him of the necessity of randomizing at 


certain stages of his particular process. His tendency is to assume that 


there are stages at which randomizing is ummecessary. If a specific potential 
extraneous cause is suggested to illustrate the need of randomizing, he 


either suggests a specific remedy in the procedure for that cause, or dis- 
misses the cause as unimportant. As other potential causes are suggested, he 


continues to suggest specific remedies or to dismiss the causes as unimportant. 
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The suggesting of specific remedies for specific causes amounts to the clas- 
sical "control of conditions". The specific remedies often add substantially 
to the technical problems and cost of the experiment, and they can never give 
full confidence in the validity of conclusions, for it is never possible to 
name all the possible extraneous causese Randomizing, on the other hand, 


even if it seems to be "fussy", is usually basically simple, and does yield 


full confidence, for it automatically neutralizes all extraneous causes. 


Dismissing a suggested cause, or other causes, as unimportant amounts to in- 
jecting an opinion into the experiment before it is performed. The funda- 
mental purpose of an experiment is not to confirm opinions, but to make the 
facts speak for themselves in the language of data. If it is worth while to 
invest time and money in experiments, it is certainly worth while to make the 
data tell the truth. 
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ACCEPTANCE SAMPLING 


A. C. Richmond 
General Supervisor, Inspection Methods Research 
International Harvester Company, Chicago, Illinois 
Fellow, Illinois Section, A.S.Q.C. 


I wonder if many of us realize the part that sampling inspection plays 
in the mass production of material. Do we realize that technically "100% in- 
spection" simply does not exist? 
Let us consider for example the production and fabrication of steel. 
The chemical properties of a 100-ton or larger heat of open hearth steel are 
determined on the average by the analysis of less than 2 ozs. 


After a heat of steel is processed into bars, shapes, etc., at the mill, 


it is shipped to a customer for forging, heat treating, machining, etc. If 
the analysis of the steel is of importance to its processing or use, it is 


usually checked for chemical analysis at the customer's plant where again a 


very small sample is used. However, the total amount of material actually 


inspected for chemical analysis ordinarily is less than one pert in two 
million, 
Many of the important steel parts that are used in modern equipment are 


heat treated; and we have specifications for their tensile strength, elon- 


gation, hardness, etc. 


Consider an important part like a connecting rod bolt 


in an internal combustion engine. Many plants will tell you that they check 


the heat treatment of these bolts 100%. But do they? In most cases, you 


will discover that they check the hardness of these bolts either by a Brinell 


hardness check on the head or a Rockwell hardness check on the side. In the 
case of the Rockwell check, compare the area actually measured with the total 
area of the bolt and again we find that the sample inspected is very small 
compared to the universe from which it is selected. 
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When we inspect parts for size, a similar condition exists. Let us con- 
sider the classification of piston pins for size. It is customary to check 
two or three diameters on each pin; and, from this small sample of the infi- 
nite number of diameters which are present in every pin, we make a decision 
as to which classification it belongs. 

I mention the above examples in order to bring out the fact that we in 
industry, whether we realize it or not, are constantly making decisions based 
on samples and, in most cases, these samples are small in relation to the 
quantity of material under investigation. Since most of our decisions are 
based on sampling, it seems obvious that we should all be interested in 
methods that tend to give us the maximum amount of information from these small 
samples, 

If I understand statistics correctly, this maximum amount of information 
from a given sample is one of the important jobs on which our statisticians 
are constantly working. Among their major contributions to industry are the 
various series of acceptance sampling tables which are becoming more and more 
generally used by our government and industry. 

My first real experience with such tables was during the last war. As 
chief of the inspection branch of the Chicago Ordnance District, on leave 


from International Harvester, it became my responsibility to supervise the 


introduction and use of army ordnance tables in the acceptance of amunition 


materiel. A short time later, these same tables were used to a varying 
degree in other branches of the procurement services. Prior to the intro- 
duction of Quality Control in the ammnition branch, a lot of data were 
collected and what was believed to be reasonable acceptable quality levels 
were established for many of the ammunition components. 


Our largest suppliers were visited. Standard sampling tables and pro- 
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36 
cedures were explained to them. On a set date, we started to use Quality 
Control Acceptance Sampling on a major part of the materiel we were accept— 
ing for the ammunition branch in the Chicago Ordnance District. That was 
mistake number one; we "bit off more than we could chew." We learned the 
hard way some of the reasons why Quality Control should be introduced slowly. 
Within a few days after we started using army ordnance tables, the rejections 
were so high that shipments were very seriously curtailed and, in some cases, 
completely stopped. Since we had been in the war for many months and this 
materiel had been found to be reasonably satisfactory, it was obvious that 
something was wrong with this new inspection system. Orders were to return 
to the old method until the trouble was found and corrected. It was soon 
found that the acceptable quality levels were set too high. I do not believe 
that this condition was directly the fault of those who set these levels, but 
rather to a chain of events. 

Prior to the adoption of quality levels, a survey was made in the plants 
of our better producers to determine their process averages. The resident 
ordnance inspectors were asked tomake a record of the number of pieces in- 
spected and number rejected for a given length of time and report the process 
average. 


Now here is what happened. Many of these operations were found to have 


lower process averages than had been thought. When results of the various 


surveys were received at Washington, I have been informed that in the inter- 
est of better quality they only considered the process average of better 
producers and so came up with a set of standards only a few could meet, 


New surveys were quickly made, more realistic acceptable levels were set 
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up in accordance with instructions that later became a part of Ordnance In- 


spection Marmal M608-8,. | 
I quote from page 7, paragraph 18, of the February, 1944, issue. "The 
. choice of an acceptable quality level must be based on consideration of two 
factors: first, the quality that is necessary for the proper functioning of 
the materiel; and second, the quality that can be reasonably expected from 
the majority of reliable producers. A balance mst be struck between these 
two considerations, and this balance must realize the necessary quantity of 
usable materiel from the producing sources which are, or can be made, availa- 
ble." 
As the war advanced and materiel became more and more scarce, we real- 
ized the importance of "...this balance mst realize the necessary quantity 
of usable materiel..." Since the war, I am afraid that many times some of 
us have been prone to think in terms of the last phrase only. 
As soon as these army ordnance tables were being used successfully, the 


quality of materiel arriving at loading plants, depots, and overseas improved 


so much that we received many favorable comments. We were able to cut our 
inspection force one-third during the time that the amount of materiel ac- 
cepted increased nearly fifty per cent. 
I have mentioned this experience in order to emphasize the importance 
of the fact that the adoption of a realistic acceptable quality level is 


necessary for the successful use of any of our acceptance tables. Unfortu- 

nately you will not find this level worked out for you in any of the litera- 
ture. I suggest that you first determine the process average of the material 
you are now using and, unless it has proven to have been very unsatisfactory, 
use it at least temporarily. 


There are several other points that mst be given consideration in order 
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to realize the most from the use of Quality Control sampling tables. One is 
to distinguish between inspection and sorting. To me, inspection is the act 
of determining whether or not material is satisfactory to use as produced, or 
presented for use. If it is found to be unsatisfactory, it can often be made 
satisfactory by sorting or some other form of rework. 

Somewhere in the instructions on the use of sampling tables, you will 
usually find words to the effect that such and such is so if the rejected 
lots are "detailed." Believe me, this is one part of the "fine print in the 


contract" you had better read. Why the word detail is used, I do not know, 


for I am sure they mean what we here in the midwest call screening or sorting. 
It is necessary to have a clear definition of inspection and sorting before 
many questions frequently asked concerning Quality Control can be answered, 

One such question is, "How mech inspection will this plan require?" An- 
other is, "What will be the quality of outgoing material which has been sub- 
mitted to this plan?" Or, in Quality Control terms, "What will the Average 
Outgoing Quality Limit be?" 

If we distinguish between inspection and sorting, the first question is 
easily answered. If the rejected lots are sorted 100% and defective items 
replaced, answers to the second question can be read directly from our tables. 

Do you realize that the average outgoing quality of anybodies' product 
is simply their process average, improved by removing defective material 
either by sorting or reworking in some other manner. I firmly believe that, 
when this fact is understood, mech of the mystery surrounding sampling in- 
spection is cleared up. You then grasp what is meant by the often heard ex~ 
pression, "Quality cannot be inspected into a product." 


When you read the literature on sampling inspection, you soon become 
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aware that the principal effect of such inspection is to force the producer 


to maintain a satisfactory process average. 


Improvement of outgoing quality 


simply by removing defects in the samples is usually very small. Unless the 
quality of his product is very bad, a producer can soon dispose of all of his 
material simply by resubmitting rejected lots. 
In the book, "Sampling Inspection," written by the Statistical Research 
Group, Columbia University, the mthors point out that even if a lot of ma- 
terial were so poor that it would be rejected fifty per cent of the time, the 
odds are 8 to 7 that it would be accepted if submitted three times. 


Since we have been emphasizing some of the shortcomings, you may well 


ask, "Why use Statistical Quality Control?" There are many excellent reasons. 


There is a risk of making wrong decisions on the quality of a lot of material 


when the decision is based on a sample. We pointed out earlier in this talk 


that nearly all inspection is necessarily performed on a sample basis. Hence, 


a risk of error is nearly always present in any inspection decision, What is 


this risk of accepting bad material and rejecting good material? When you 


use statistical methods, these risks can be closely calculated. The amount 
of inspection necessary to assure you that any given risk is not exceeded may 


be determined. You are then in a position to approach more intelligently the 


economical balance between cost and perfection. 
Another important reason for using statistical tables is that we soon 
learn protection is not based on per cent inspection, but is closely related 
to the size of the sample. This is a hard fact for the "ten percenters" to 


grasp, and I was one of them until a few years ago. When we consider that 


the information on which a decision is based is contained in the sample and 
that the size of the lot from which it was taken does not change the amount 
of information in the sample, it is easy to recognize the fallacy of per 
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centage inspection. 

The question is sometimes asked as to why it is necessary to have so 
many tables. Before any table is selected, we should be able to answer two 
questions. 

1. What is the worst quality that we can afford to reject a small per 

cent of the time? 

2. What is the worst quality that we can afford to accept a small per 

cent of the time, 

In industry, acceptable quality levels vary and the necessary risk of 
wrong decisions vary. For each combination of AQL and risk, a different 
plan is required, hence the need for many tables. Series of tables that 
cover nearly all ordinary requirements have been computed and published. A 
series of tables now rapidly gaining in use are the new "JAN" (Joint Army 


Navy) tables. The original Dodge-Romig tables and Army Ordnance tables con- 


tinue to be very generally used. 

This discussion leads us up to "Operating-Characteristic Curves." For 
every sampling plan, a curve can be drawn which shows the per cent of lots 
that will be accepted or rejected, on the averare, for any everage per cent 
defective of submitted lots. I have here (Slide No. 1) a characteristic 
operating curve selected from the Navy Inspection Manual Appendix X. This 
curve is drawn for a single sampling plan using a sample size of 75, an ac- 
ceptance number 2, and a reject number 3, an AQL of 0.65 to 1.2 and an AQQL 
of 1.5 to 2.5. To the uninitiated, this curve may look complicated; but 
with a few definitions and a little explanation, it is quite easy to read 
and understand. 

AQL, "Average Quality Level," in these Navy tables is that quality ex- 
pressed in per cent defective which we are willing to accept 95 per cent of 
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the time. The numbers on the left of this curve denote the percentage of 
times a submitted inspection lot will be accepted. The numbers across the 
bottom represent the percentage of defective items in submitted inspection 
lots. 

If we enter this chart from the left dt 95, extend across to the inter- 
section with the curve, and then drop down to the bottom line, we find the 
nearest per cent defective to be 1. This cane that, if the average per cent 
of defective items in the inspection lots submitted is 1, approximately 95 
per cent of the lots will be accepted. 


Now suppose that we want to know how bad the lots would have to be be- 


fore we would reject nine out of ten of them. We simply enter the table from 
the left at 10, proceed across to the curve and down to the bottom line where 
we find the per cent defective to be about 7. This inspection plan will 
therefore reject on the average 9 out of 10 inspection lots submitted which 
cneihaiin on the average 7 per cent defective items. In the same manner, for 
any average lot quality, the per cent of lots accepted or rejected can be de- 
termined. 

Provided all rejected lots are sorted and the defective pieces are re- 
placed with good ones, it is easy to determine the Average Outgoing Quality 
(A0Q) and Average Outgoing Quality Limit (AOQL) from the Operating Gharac- 
teristic curve. By AOQL is meant the worst average per cent defective 
quality of outgoing product that could pass through, regardless of the 
quality of the lots submitted for inspection. 

The Average Outgoing Quality for any average per cent defective sub- 
mitted lots can be determined as follows: As an example, we will take one 
that is easy to figure. Suppose that under this plan a producer is submit- 
ting material that averages 3.25 per cent defective; from the curve we de- 
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9. 
termine that fifty per cent of his lots would be rejected. If all defective 
material is removed from one-half of the lots and replaced with good parts, 
one-half of our outgoing material will be 3.25 per cent defective and the 
other half will contain no defective items; therefore, our outgoing quality 
would be approximately 1.6 per cent defective. Similarly, the outgoing qual- 
ity can be determined for any other average per cent defective of submitted 
lots. 


Under any inspection plan, provided the inspector makes no errors, all 


lots will be accepted when the material is 100 per cent good and all lots 


will be rejected when the material is 100 per cent bad. In between these two 
extremes, a sampling plan will accept less and less material as the product 
becomes progressively worse. 

From the Operating Characteristic curve, another curve called the Aver- 
age Outgoing Quality (A0Q) curve can be plotted. This curve is obtained by 
using the outgoing quality levels and incoming as 
ordinates. This curve starts at O per cent defective, rises rapidly to a 
peak, and then falls away rapidly until near 100 when it flattens out and 
reaches the base line at almost a tangent. The peak of the curve is the AOQQL. 
I mention this curve as a means of taking some of the mystery out of one of 
the terms frequently used when discussing statistical sampling plans. 

There are many interesting curves and formulae used in Statistical 
Quality Control; and, to those who like mathematics, it offers a challenge 
to both the professional statistician and the high school graduate. . 

During the last of the war, several International Harvester men attended 
some one of the eight-day courses in Quality Control which were authorized by 
the Office of Production Research & Development of the War Production Board, 
The successful results of Quality Control applications in two or three of our 


plants attracted the interest of our top management. 

Upon my release from the Ordnance Department late in 1945, I was given 
the job of making recommendations in regard to the general use of Statistical 
Quality Control in International Harvester Company plants. The first thing a 
study of the problem brought out was the need for an educational program. 

The following plan was submitted to, and approved by, our top management: 

1. A two-day meeting attended by all our Works Managers be held on 

Statistical Quality Control. 

2. A ten-day intensive course in Statistical Quality Control be held 

for Chief Inspectors and other selected men from each of our Works. 

Dr. Lloyd A. Knowler consented to assume the responsibility of preparing 
the course, and with the aid of Drs. Irving Burr, Edwin Olds, and Mason 


Wescott presenting it to the class. Professor Knowler and Professor Olds did 


a splendid job of selling Quality Control to the Works Managers with the re- 
sult we had a class of 70 selected men on hand for the ten-day course held in 
August, 1947. 

Also present during the ten-day course was one man from the Accounting 
Department of each of our divisions representing the Division Comptroller. 
These men have been of much value when it comes to presenting to management 
the savings resulting from Quality Control. With the four professors we just 
mentioned instructing our class, it is needless to say that it was very suc- 
cessful. Many of the men from this class have become Quality Control Engi- 
neers and fill other key positions in our Quality Control program, 

In most of our plants, Quality Control is under a "Quality Control Engi- 
neer" who reports directly to the Chief Inspector. In most plants, we now 
have a "Quality Control Action Committee." The Chairman of this committee 
is usually a Staff Assistant reporting directly to the Works Manager who is 


10. 


the highest official of the plant. The committee, in addition to the 
chairman, generally consists of the Chief Inspector, Quality Control Engineer, 
Planning Engineer, and Master Mechanic, In this group are the men who can 
take immediate corrective action when required. These "Action Committees" 
have proven to be one of the most valuable contributors to our Quality Con- 
trol program. 

In many of our International Harvester Company plants, we are using 
statistical sampling tables for inspecting material between departments and 
on inter-works shipments, as well as on material received from outside 
vendors, 

It so happens that our plant here in Milwaukee was one of the first 
International Harvester plants to become interested in Statistical Quality 
Control. Nearly all material produced in this plant is now being inspected 
in accordance with standard inspection tables. Rejected lots are returned 
to the department at fault for rework or sorting. The results are very 
gratifying. 

The next two slides are reproductions of actual charts which are typical 
of the "before" and "after" effects of statistical sampling. This first 


chart (slide No. 2) is a per cent defective (p) chart. It shows the per cent 


of defective items found in samples at the time statistical sampling was 


started on nozzle valve guides used in our Diesel engine injection pumps. 
These pumps are built at our Milwaukee Works. 

Those of you who are familiar with Quality Control will notice that this 
same chart form can be used for X-R (average and range ) charts. In the space 
ordinarily used for plotting ranges, Milwaukee Works lists the characteristics 
checked. Opposite each characteristic is shown the number defectives found 
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in each sample. In the upper chart, the per cent of defectives found to the 
number of pieces in the sample is plotted. 

The sub-group size is 700 which corresponds to one shift's production. 
The sample size is 75, acceptance number 2, rejection number 3. You recall 
that this is the same sample size and the same accept and reject numbers used 
in the operating characteristic curve we were just discussing; therefore, the 
AQL and AOQL are the same. In use, the shifts are designated by different 
colors. On these slides, they are shown as indicated. 

Prior to the time this chart was started, these nozzle valve guides were 
100 per cent screened by the Inspection Department. Rejections were high, re- 
sulting in a considerable amount of scrap and rework loss. In about three 
weeks! time, you will note that a big improvement was made in their process 
average. The process average was still not satisfactory, resulting in the re- 
jection of too many lots. Examination of the chart shows that if the runout 


specification could be held, more than 50 per cent of the defectives would be 


eliminated. This problem was discussed with the design engineers, resulting 


in a change in the specifications from .002" to .003" permissible runout. 
This change you see was noted on the chart January 17. 

A real effort was then made to bring the other characteristics under 
control. The results are shown on the next chart (slide No. 3). Note that 
a large majority of the samples are now free from any defect, and it is 
seldom necessary to reject a lot. Results: better quality, less scrap, less 
rework and inspection cost. 

The next slide (slide #4) shows crank-cases for our Silver Diamond 
engine being inspected. The Silver Diamond engine is built at our Indianapo- 


lis Works and is used principally in the smaller line of International trucks. 
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These cases are checked for 22 characteristics in accordance with a Dodge- 


Romig double sampling table for 2 per cent AOQQL. The table used is for lots 
of 51 to 100 and calls for a first sample of 23, a second sample of 23, with 
an acceptance number of O and a rejection number of 2. On account of the 
large number of characteristics checked, this is probably the most difficult 
item now being inspected in accordance with standard sampling procedure in 
our company. 

They have an interesting method of making sure that the sample is chosen 
from each lot in a random manner. The crank-cases are numbered from 1 to 100 
as they are received at the inspection bench. Four cards have been prepared, 
each card has 23 random numbers from 1 to 100. When a new lot starts to ar- 
rive, the cards are shuffled and two are drawn. The numbers on the first card 
become the first sample; and the second card, the second sample. Our con- 
veyors are so arranged that the first samples pass the inspection benches 
immediately, and the second sample is shunted onto another conveyor. The re- 
mainder of the lot is held until final disposition of the lot is made, 

The results have been exceptionally good. Under the old so called "100 
per cent" inspection method, about 5 per cent of the crank-cases caused 
trouble on the assembly line. Now less than .5 per cent cause trouble. For 
approximately the same production, 17 inspectors were required under the old 
plan; now only four wre required and, by simply increasing the lot size, it 


is estimated these same four men will be able to handle a 50 per cent in- 


crease in production. Many other items at this Works are now being inspected 


according to standard sampling plans. We are informed that Indianapolis 
Works expects to use them almost 100 per cent in the near future. 


This Works has adopted a novel plan of grading its vendors. Nearly 
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all incoming material is inspected on an acceptance sampling basis, and the 


vendors! process average is computed for both major and minor characteristics 


on the basis of the samples of their product inspected each month. They are 


then given a "Quality Rating." 


This rating is made in accordance with a key 


sheet which is reproduced on the next slide (slide No. 5). Notice that the 
first eight ratings are in equal increments and the last includes all over 
the eighth. 


Each vendor is notified of his monthly Quality Rating. In addition, on 


a large board in the inspection office, each vendor's previous monthly rat- 


ing is posted. You can easily guess that this board has become of interest, 


especially to competing vendors. The results of standard sampling and vendor 
quality rating have been very good. The composite ratio of defective items 
received decreased approximately 20 per cent during the first six months and 
the trend is still downward. 
Statistical Quality Control has two main divisions: "Process Control" 


and "Acceptance Sampling." Acceptance sampling is divided into two groups: 


inspection by "Variables" and inspection by "Attributes." The first is a 
method of measuring and analyzing the variation of items in samples and be- 


tween samples. 


In inspection by attributes, each item is classed as ac- 


ceptable or not acceptable without regard to degree of non-conformance; for 


example, items checked with a "go, no go" gauge. This talk has been limited 
to e discussion of inspection by attributes. 


In the International Harvester Company, we are using all of the comonly 


used Statistical Quality Control Techniques. In the fiscal year 1949, 
Quality Control was credited with being one of the major contributing factors 


which not only resulted in a major improvement in the quality of our product, 
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but also resulted in a savings in scrap, rework, and inspection cost of over 


two million dollars. According to present trends, a similar amount will be 


realized this year. 


SLIDE NUMBER 1 


OPERATING CHARACTERISTIC CURVE 
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PERCENTAGE OF DEFECTIVE ITEMS IN SUBMITTED INSPECTION LOTS 
TAKEN FROM: NAVY DEPARTMENT STANDARD SAMPLING INSPECTION TABLES— APPENDIX X 
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Inspecting Silver Diamond Crank-cases 


Indianapolis Works 
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MAJOR CHARACTERISTICS 


PERCENT DEFECTIVE 


RATING NO. 


1 
2 
3 
4 
5 
6 
7 
8 
10 


.00 - .50 
-51 - 1.00 
1.01 - 1.50 
1.51 - 2.00 
2.01 - 2.50 
2.51 3.00 
3.01 3.50 
3.51 - 4.00 


4.01 AND ABOVE 
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QUALITY RATING 
MINOR CHARACTERISTICS 


PERCENT DEFECTIVE 
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4.51 - 6.00 
6.01 = 7.50 
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INSTALLING A COMPLETE QUALITY CONTROL SYSTEM 


by 
S.C. Amren <= Director of Quality Cmtrol 


U. S. TIME CORP, 
Fellow - Southern Connecticut Sectim ASQ 


The task of installing a complete Statistical Quality Cmtrol Program in 
any sizable industry presents many problems and consideratios. Top manage 
ment naturally expects returns on it's investments and S. Q. Ce should be 
considered as such. From this investment is expected better operating effi- 
ciency, lower scrap and rework losses, reduced inspection costs, and better 
customer relations. Unfortunately, S. Q. Ce is not something that can be 
purchased in a package ready for use, but requires considerable perseverance 
for success. Some industries have failed to realize a profit from it's in- 
vestment in S. Q. Ce and invariably this failure can be traced to a lack of 
wholehearted cooperation m the part of management. 

The problem from the outset is for the Quality Manager to sell manage- 
ment thoroughly enough so that there is some em thusiasm behind the movement 
all the way dow through the lines of supervisim. As som as this has been 
accomplished, he must evaluate the overall situatim and attempt to direct 
his efforts toward a successful goal. Probably his initial cmsideratims 
should be given to at least all of the following. 

1. Does he have the necessary technical personnel to administer 

the program? 
What is the present cost of inspectim? 
What is the calibre of the present inspectiom force? 


What is the preset cost of scrap and rework? 
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Se What is the cost of servicing and replacement of field returns? 

6. Does the inspection department have adequate measuring equip- 

ment and gaging? 

7- How should the training program be undertaken? 

Consider the above items in some detail. 

le Does he have the necessary technical personnel to administer the 

program? 

The success of a Quality Control program providing it is given the proper 
backing depends largely on the individuals who are to administer it. These 
individuals should be thoroughly trained in S. Q. C. techmiques and should 
have either an engineering or statistical backgrowmd, preferably both. If 
the industry is large emough to warrant several such men it would be well to 
obtain a mixture of Ingineers and Statisticians as both such backgrounds are 
seldom fownd in one man, It is also helpful if these men are familiar with 
the preducts and processes involved and possess personalities which are cm- 
ducive to cooperation. 


2. What is the present cost of inspection? 


It is important that cost of inspection ibe determined accurately using 


some reference. Probably the most common references are percent of direct 
labor and inspection cost per mit. It is advisable to obtain both as some- 
times S. Q. C. reduces direct labor costs per wit more than it reduces in- 
spection costs, and consequently the inspection cost shown as percent of 
direct labor will be wfavorable. If, however, you have figures showing in- 
spection cost per wit you may avoid undue criticism. This information should 
be charted every week or mmth as the program progresses. 

3. What is the calibre of the present inspection force? 

It is not uncommon to find Inspection departments filled with older 


people who can no lomger keep pace with mass production lines. If this is 
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the case, the program is doomed unless corrective acti is taken. S. Q. C. 
changes the entire concept of inspectim, and it is much easier to change it 
in a yowmg progressive mind than it is in someone who has become accustomed 
to old fashioned methods over a long period of years. It is essential that 
inspectors have the ability to wderstand at least the basic principles of 
Quality Control methods, since lack of this ability will only cause them to 
disregard the sampling plans, and consequently no benefits can result. 

he What is the present cost of scrap and rework? 

Another way of showing progress of an S. Q. C. program is by charting 
scrap and rework. It is important that accurate figures be obtained to 
show these losses before the program gets under way, since without this 
picture it may be difficult to substantiate the investment that maagement 
is making. It is not sufficient, however, to obtain scrap figures alme; 
it is also important to get production figures so that scrap and rework may 
be shown as a percent of good worke 

S. What is the cost of servicing and replacement of field returns? 

No product however good is free from defects and most industries carry 
a Customer Service Department to handle complaints and repair or replacement 
problems. The cost of maintaining this department plus replacement and 
servicing costs are items that Quality Control will reduce. It is advisable 
to work closely with this group as much valuable information is often found 
in it's files. 

6. Does the inspection department have adequate measuring equipment 

and gaging? 


Invariably, an S. Q. C. installation wcovers some instances of in- 


adequate gaging and measuring equipment. This is one of the major cmtri- 


butions that these techniques make to inspection departments. Quarrels and 


misunderstandings brought about by questionable gaging are no longer necessary. 
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There is no way of positively assuring ourselves that a product meets speci- 
ficatims wmless the measuring equipment being used has been statistically 
proved to be adequate. the other hand, measuring equipmmt has sometimes 
been condemned when it has not been at fault. Such questios will be answer- 


ed by statistical teclimiques. 


7e How should the training program be wdertaken? 

The problem of training the people in various capacities in the organi- 
zatim is a large but necessary task. Probably the first group to receive 
this training should be the key inspection personnel. However, it soom be- 
comes apparent that production supervisors and methods engineers need the train- 
ing almost simultaneously with inspectim personel. Then as the program pro- 
gresses into the machine capability stage and revision of some tolerances are 
indicated, it will be necessary to enlighten design engineers about the tech- 
niques being used. It is also advisable to acquaint your sales force with 
the new program so that they may take advantage of the more reliable quality 
aspect in talking with customers. 


The foregoing has dealt with some of the earlier aspects that must be 
considered in an Se Qe Ce program. However, this is mly the beginning and 
the most difficult hurdle is with the production employees themselves. A 
great deal of effort should now be expended in selling quality workmanship 
to these people. This can best be done by the Quality Cmtrol Mngineers, 
who are in cmstant touch with production. There will be occasions where an 
S. Qe Ce. application will show some startling results. The employees con- 
cerned with this improvement should be given more than their share of woth 
as instances of this nature will do more to promote the program than anything 
else. Be sincere with the employees, sell the program to the mim, make 


everyme concerned feel the importance of quality workmanship for his om 
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security. Remind them often that quality makes sales, sales make jobs. Do 
not hide scrap figures but show them how much is being lost for this reasm. 

Most employees like to do good work providing someone at the supervisory 
level is showing an interest in what is going m. Quality Control charts af- 
ford a real opportunity for the supervisor to keep posted m what is going on 
with a minimum amowt of effort. In cases that are out of cmtrol due to oper- 
ator inefficiency make sure the operator knows what he is supposed to do and 
that he has the means of doing it properly. 


EFFECT OF S. Q. C. ON VARIOUS INDUSTRIAL FUNCTIONS SUCH AS METHODS, DESIGN 
ENGINEERING, PRODUCTION, SALES, PURCHASING AND INSPECTION. 


METHODS 

The major cmtribution that S, Q. C. makes to the methods department is 
in showing process capabilities. This is a great aid to the Methods MIngineer 
since it is now possible to accurately establish the fitness of any machine 
or process for its intended job. 

DESIGN ENGINEERING 


Design Ingineering embodies the function of setting tolerances and estab- 
lishing specificatios. Often in the past these have not been realistic, 
causing undue hardship on both methods and productim. With S. Q. Ce the 
design engineer now has available to him a wealth of information ™m what can 
be expected of men, machines, and material. He is thus in a better position 
to establish realistic specifications for the best efficiency and quality 
balance. 


PRODUCTION 


This function probably bears the brunt of the attack by Se Qe Ce How- 
ever, it also realizes more benefits than any other function providing prod- 


uction supervisors take advantage of the information being made available. 
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Wealmesses of almost any nature are pointed out and corrective action results 


in substantial savings in improved efficiency, scrap, rework, and better qual- 
ity. 

SALES 

The sales organization is always looking for high quality to keep custom- 
ers happye Se. Qe C. is without a doubt the best ally that any salesman ever 
had since he is assured that the Quality angle is being cmtrolled scienti- 
fically and with much greater dependability. He is proud of this fact and 
uses it to good advantage on his customers, Furthermore, an increasing number 
of purchasing agents are asking vendors if their product or material is statis- 
tically cmtrolled before buying. 

PURCHASING 


One of the functions of the purchasing department is that of supplying 
the industry with raw material, parts, and various other items at the most 
advantageous prices. It is sometimes difficult for the purchasing agent to 
know whether or not he is getting a bargain. S. Q. ©. affords accurate com- 
parisons of vendors as well as more accurate information about the material 
being purchased. 

INSPECTION 


The function of an inspection department is to assure management that 
the product being made conforms to specificatios and acceptable standards 
of workmanship. This has always been the function of the inspection depart- 
ment, but not wmtil the advent of S. Q. Ce has it been possible to adhere 
to ite Inspectim might better have been described as the department that 
did all the sorting, thus covering up the sins of the production departments. 
Under modern techniques, inspection does no sorting whatsoever. This must 
be dme or at least ee charged to the department making it necessary. Sort- 


ing is no more a function of inspection than rework is, since both are making 


the material fit for use. Why then should&'t it be paid for in the same 
mann e 

This also holds true for purchased material. If purchased material 
is wmsatisfactory there are three alternatives. 

l. Reject the material. 

2. Sort and reject the defectives. 

3. Accept the material with production taking a variance. 

The first of these is the ideal, but wmfortunately due to urgent situat- 
ions it can not always be dme. If it is necessary to resort to 2 or 3, the 
sorting costs or variance incurred should be charged to purchasing. Purchas- 
ing may or may not be successful in recovering part or all of the costs from 
the vendor and in the event that they are not recoverable they should be 
charged to the original purchase cost. In this way the true cost will be 


apparent and it may be more ecmomical to buy a better grade of material. 


THE PROGRAM AT U. S. TIME CORP. 


It is now 15 months since we began our S. Q. C. program at U. S. Time 
and we have followed the thinking outlined in the foregoing quite closely. 
As Director of Quality Control I report directly to the president of the 
company and am respmsible for all S. Q. Ce. and Inspectim activities in 
all four of our plants. We have beam given complete backing, from top manage- 
ment, in our efforts with the result that our progress has been quite rapide 
We have gained considerably in productim efficiency, better quality, re- 
duced scrap and rework, etc. but it is still a little early to determine 
what the effect will be on our customer service costs. We believe, however, 
that a large portim of the present cost of the entire quality fmctio will 


eventually be paid for through the reduction of these costs. 
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This company manufactures a variety of inexpensive watches, alarm 
clocks and various other timing devices. Because of the production volume 
and specification requirements on watches we have directed most of our atten- 
tion to this item. 

As our training program got under way it soon became obvious that the 
educational backgroud of most of our Inspectors was such that statistical 
techniques could never have been absorbed. This was a major obstacle but 
some sort of action was necessary, and consequently the problem was dis- 
cussed with our Director of Industrial Relatioms. It was finally decided 
to discuss the matter with our wion and make a proposal that we would give 
an aptitude test to all employees who had Inspection experience, The test 
was to be in two parts; one for straight arithmetical ability and the other 


for intuitive ability. The wion agreed to the proposal and notices were 


then sent to all who had inspection experience including those who had been 


laid off due to production cutbacks. Approximately 250 candidates turned 
out and out of this group 55 qualified. These were immediately put to work 
and those who failed, but had enough seniority were transferred to various 
jobs in the production and repair departments. We have since found it 
necessary to hire additimal inspectors, but have insisted that these men 
pass this same test and have had inspection experience, As a result of 
these steps we now have a high grade progressively minded inspection depart- 
ment, and this has contributed materially to our progress. 

An additimal investment was required for measuring equipment since 
some of these items were found wholly inadequate. We now have some of the 
finest in the world and even the best is none too good cmsidering the tol- 
erances we must cope with. Our gaging is cmtinually being improved to fit 
Se Qe C. raquirements. 


Early in our program we were searching for a difficult and trouble- 


some process to which we might direct our efforts and gain some prestige 

with production. The production manager was happy to point out a screw 
machine job requiring 6 hours of machine time per day plus the time of 

6 girls to sort the parts, whereas 16 hours of machine time should be suffi- 
cient providing a 100% yield were obtained. We immediately began a series 

of machine capability tests using X and R charts om the dimensions causing 

the difficulty. It was soon obvious that the machines being used could not 
yield 100% with present tolerances. We then had our first encowmter with 
Ingineering in regard to tolerances. We explained to them that by increasing 
the tolerance in such a direction that on the average we would have a closer 
fit between the part in question and the mating part. gineering was quite 
concerned that some of the parts would not go together since this permitted 

a line to line fite After considerable explanation of the laws of probability, 
the change was granted with the result that we now have only i6 machine hours 
per day and no sorting whatever. Furthermore, the assembly department is 
pleased with the better average fit. This application did give us a tremendous 
boost with the result that several more jobs of similar nature were called 

to our attention. Machine capability tests are cmtinually being rum om new 
jobs so that discrepancies like the above can be ironed out before any damage 
can be done. No new jcb is permitted to ru on a production basis wtil our 


capability tests indicate that it can be made to specifications. 


Out inspectio plans in the various departments are tailored to fit the 


requirements of each group of machines. In departments, such as Automatic 
Screw Machine, Pinion Cutting, and Secondary Operations, we employ three 
different plans. 
1. Acceptance sampling of the @mtire days production. 
This type of plan is the most economical and is used where very 


little difficulty is encountered in meeting print requirements 
or where operators are extremely capable and scrap is seldom 
produced. The sample size of course depends upm the lot size 
and the particular requirememts of the part. 
Process Inspection based on attributes. 
This type of plan is employed where more cmstant supervision 
is required and scrap could readily be produced. The inspector 
makes a complete round every 2 hours and takes a first sample of 
10 with c = 0, If c = 2 or more, the producti during that two 
hour period is scrapped. If c =1, a second sample of 20 is 
taken and if any additional defects are found, the lot is scrapped. 
If no more defects are found the lot is accepted. This plan may 
seem lenient mathematically, but we feel it is a better balance 
between quality and production than our first plan, wherein the 
sample sizes were 20 and 0 and using the same "c" values. 
Furthermore, our present system permits the inspector almost 
twice as many rounds per day which is also desirable. 
Variables sampling. 
There are always certain dimensions of critical nature and 
where machine capability tests indicate that the machine is 
barely able to meet print requirements. It is necessary in 
such cases to resort to X and R charts to assure that no scrap 
is being made and to indicate in advance when machine settings 
need be changed. 

The amount of Inspection required in departments such as the above 


depends on the ability of the operators and the number of machines they are 


required to rune It is folly to load an operator down with so many machines 


that he does not have time to keep checking the work beirig produced. Poor 
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operators or operators who are loaded dom with too many machines require 
more inspection, thus creating an incorrect and unjust picture of direct 
and indirect labor charges. 

Our sub and final assembly departments require the same kinds of 
plans as those used in our machine departmamts but to a lesser extent. 
There are a few processes requiring X and R charts because of their critical 
nature, but on the average the cost of Inspection in this area is much less 
than in our parts manufacturing areas. 

We have made serious efforts to keep- our Inspection costs to a mini- 
mum and still maintain the degree of cmtrol necessary for the most economi- 
cal balance. It is interesting to note that in order to accomplish this it 
has been necessary to supply an inspection force to our parts manufacturing 
departments which is equivalent to 30% of direct labor. However, the cost 
of inspection in our sub and final assembly departments is less than 10% of 
direct labor giving us an overall average of 15%. To some, this may seem 
high but it should be borne in mind that our parts are extremely small with 
very close tolerances and consequently, difficult to inspect. Furthermore, 
it is to be expected that these percentages will be reduced as our program 
progresses and operators become better acquainted with these new methods. 
The ultimate will only be reached when operators become capable enough so 
that acceptance sampling can be used almost exclusively. 

Since we purchase a variety of parts from outside sources it is also 
necessary to assure ourselves that these compments satisfy our print re- 
quirements as weli as those made in the plant. All such parts are sub- 


jected to an acceptance sampling plan which varies with the requirement 


of each particular part but rarely is in excess of 2% Ordinarily, the 
parts are rejected if they exceed our established percent defective but 


we do have situations where due to emergencies it is necessary to sort them, 
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In cases of this nature we are careful in seeing to it that costs thus in- 
curred are charged to purchasing. If these costs can not be recovered 
from the vendor, then it is mly logical that they should be added to the 
original cost. In this way purchasing is in a better position to evaluate 
the various suppliers from the cost versus quality standpoint. 

Our Sales organization has been very interested in the progress of our 
program and we have attempted to give them as much informatim as possible 
on the improvement in quality. Furthermore, we have rum tests on competitive 
products and evaluated the results to determine our own wealnesses as well as 
our competitors strong points. Recent comparisons indicate that we now have 
the highest quality timepieces within our price range and we intend to keap 
ourselves in this position by a progressive program searching for continued 
improvement. 

The results we have obtained thus far are substantial and quite grati- 
fying. The percent defective from our screw machine department has been 
reduced from more than 5% to less than 2%. The production on one of our 


watch movement lines has been increased by 76% during the past year. This 


improvement was partially due to an incetive plan but the overall improve- 
ment could not have been realized without the improvement in parts the line 
is now getting. The percentage of watches from the movement assembly line 
accepted for the first 2) hour run has increased from 76% to 93.3%. The 
percentage lost during the 2); hour run has been reduced 28% and the total 
lost in the dialing and casing sectim has been reduced 5%. More important, 
however, is the improvement in quality of product as it leaves the plant. 
According to our Quality Analysis, the percentage of watches containing 
critical or major defects has been reduced from approximately 5% to less 
than 1%, We believe that this quality level is the best that has ever 
been achieved by any manufacturer of inexpensive non jewelled watches. 
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This quality level should reduce our customer service and repair expenses 


considerably and help to defray the cost of our program to a great extent. 


SUMMARY 


When installing a Quality Cmtrol Program the first requirement is a 
complete picture of the present status in regard to scrap, rework, inspection 
cost, customer repair service, and manufacturing efficiency. Then a thorough 
job of selling the program to top management and all department heads that 


will be affected is the next requirement. In selling the program it is 


necessary to do some educating sw that the individuals concerned have at 


least an understanding of the basic fundamentals involved. Be sure that 

the inspection department is mentally capable as these techniques require 
some arithmetical ability. Select and train qualified men as Quality Control 
Fngineerse Proceed with the program cautiously tackling the most trouble- 
some situations first. Take advantage of spectacular cases to further the 
cause but be sure to give credit to production wa kers when due. See to 

it that corrective action is forthcoming where cmtrol charts indicate the 
necessitye 

Make every effort to show methods and design engineering the power of 
these new techniques as these function more than anyone else will appreciate 
the factual informatio made available through process and machine capability 
tests. 

Keep up to date charts showing the progress that is being made with 
regard to scrap, rework, inspection cost, customer repair service, manufactur- 
ing costs and outgoing quality levels. These should be reviewed periodically 
with top management as well as all supervisors concernede 
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When the program is well wder way and results substantial it is 


advisable to prepare a report for top management showing in dollars and 


cents what savings have been affected. 
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RESULTS OBTAINED DURING FIVE YEARS 
OF OPERATION IN A MAIL ORDER PLANT 


James Baellowe* 


Statistical quality control can be applied to eny kind of a 
process in which there is repetition. Because of this, a mail order 
plent is @ statistical mecca, With many thousands of orders eech dey, 
and with the processes broken down into activities so that eech employee 
does just a certain amount of work on the order and no more, there is 


a situation which lends itself to statistical quality control. 


Most quality control applications pertain to machine opere- 
tions. While statistical formulss and quality control charts are 
availeble, not many epplications have been reported on purely clerical 


and wanuel work. As distributors, we do no manufacturing. Clerical 


end manual operations are the only kind we have. In introducing the 
progrem, therefore, we were faced with some of the practical problems 


which always srise in the edeption of e new management technique. 


Since statistical quality control hed been used so success- 
fully on machine operations, my original thought in the fall of 1944 
when I was taking the first 10-day intensive course in the subject 
at the State University of Iowa, was to draw an anslogy between machine 
operations end human operations. While one dislikes to compere humans 
with machines, we might recognize thet some similar cheracteristics 
exist as shown by the following sneslogy: 
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Mechine Operat Human Operetions 
Scrap (or rework) fhrowbacks (for correction) 
Repetition of defective perts Repetition of same type of error 
Specifications Goels 


Mejor and minor defecte Errors effecting end errors not 
effecting the customer 


Introductory Problems | 
Por any management technique to be epplied successfully, 


there sre usually prectical opereting problems which have to be solved. 
I knew the following hurdles would heve to be overcome if statistical 
quality control methods were going to be applied successfully in our 
plant. 


1. The simplest type of mathematics. While it is peoete 
to imagine statistics without any methematics, I knew thet 
the department managers and supervisors who would operate 
these nev control methods were not mathematiciens. Nor do 
they heve the time to work out certain necessaery techniques 
such as control limits. To overcome this objection, we pre- 
peared e quick-reference teble as shown on pege 3. After a 
manager hes observed his error ratio, he merely hes to look 
at this table to determine his control limits. 


The control chert would heave to be simple, easy to under- 

stand, and quick to comprehend. We prefer to do most 

things “at a glance". 

The charts would have to be practical from en upkeep point 

of view but attractive in eppearance. Nothing creates ea 

better atmosphere in en operating depertment then charts 

which ere kept up, and which are neat and sttrective in 

appearence. 

All of the above problems vere solved in meking our epplicetim 
of statistical quelity control. After I describe our procedures and 
@xplein how we maintain our control cherts, you will readily see how 


we were eble to solve the practical problems I heve enumersted. 
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VALUB OF THE CONTROL LIMITS OF AN np-CHART CORRESPONDING TO VARIOUS 
VALUES OF AVERAGE NUMBER OF ERRORS MADE PER 100 WORK UNITS 


ENTER LINE UPPER CORTROL LIMIT | CENTER LINE UPPER CONTROL LIMIT 


Average (2Sigme Limit) (Average (28igme Limit) 

Number of Number of . 
Errors per ' Errors per 

100) 100) 


.10 2.10 4.97 
-20 1.99 2.20 5-15 
“ho 1.66 
"50 1.91 2.50 5.62 
.60 2.14 2.60 5.78 
"70 2.31 2.70 5.98 
80° 2. 2.80 "10 
"90 2.79 2.90 6.26 

1.00 2.99 3.00 6.41 

1.10 3.19 3.10 6.57 

1.20 3. 3.20 6.72 

1.30 3.57 3.30 6.87 

3.75 7.02 

1.50 3.93 3.50 7.18 

1.60 4.11 3.60 7.33 

1°90 3°90 

2°00 4:80 7.92 


THE LOWER CONTROL LINTS is sero for each of these cases. 


Aldens Quality Control 
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The Alden Control Chert 

The up chert, used et Aldens, measures 15 inches in height 
ané 10 feet in length. It is prominently displeyed in the center of 
the activity being controlled es shown in the photograph on the next 
pege. On the bottom of the chart we show the days of the month with 
sufficient space for each dey to sallow for about six plottings. In 
the left margin of the chart the number of errors is indicated, from 
0 to 13. Parallel lines running horizontally across the entire chart 
aid in observing the number of errors found in the semple. 


Quality Control Chart — Div. M 
Humber of Frrers per Tikets 


-$- 


Marl marti mar dl Marli mer 30 


a 


™ 
25 6784 13 4 Bl 2823 27: 


Pigure 1 
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The sample sise is 100 work units in ell ectivities. In 
some departments the vork unit is a customer's order; in the merchandise 
Gepartments it is called a "ticket"; in the Shipping Roce the work 
unit is an entire peckege. The edvantage of using a sample sise of 
100 work units is that the error ratio conteined in the sample is 
easily computed and reedily comprehended. 


In meking the plottings on the chart we use circuler gummed 
tabs one-half inch in diemeter. These are affixed to the chert on 
the horisontel line which indicates the number of errors found in 
the sample lot inspected. We then superimpose on the teb a gummed 
numeral indicating the number of errors the line represents. These 
tabs are connected by means of red gummed peper tepe 3/16 of an inch 
in width, easily effixed in the seme manner. This construction is 
indicated below: 


Figure 2 
By using these meterials, 411 of the charts ere kept uniform end 


ettractive in eppearence. Anyone who can apply a postege stemp 
een make the plottings. 
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We maintain the charts on a monthly besis. On the first 
day of each month we put up a fresh chart in each department, using ea 
steple gun. The sheet is affixed to « permanent beckboard, In the 
upper right corner of the chart we show the error retio established 
for the preceeding month. In the upper left corner we show the 
upper control limit applicable to thet particular ectivity. So long 
as the plottings remein below the upper control limit, with reasonable 


variations, we heve confidence that the process is consistent. When 


@ plotting fells outside the upper control limit, however, a directive 
from top menagement requires that the depertment manager personelly 
inquire into the result to see whether there is ean assignable ceuse-- 
whether a particular operator or type of error is responsible. If 
the plottings stay within the control limits, we do not require thet 
the manager give attention beyond his normel review of production 
results. 


A solid line, extending ecross the chert, shows what the 
goel is for the month. Ia setting « goal we consider recent experience, 
the possibility of major volume fluctuations, and the results being 
obtained on compareble operations in other departments. We advise 
ageinst setting « goal when a chart is first placed on ea new activity. 
At that stage the important thing is to find out how the process is 
really running. We have gone es long es two months before setting 
a goal. But efter sufficient deta is obteined, « reasonable goal, 
possible of attainment, should be set. 


In the meil order business there ere wide volume fluctuations, 
perticulerly pricr to Christmas and Eester. Therefore, it is 
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advisable to set a goal which will make a reasonable allowance for 

veriations. The goal may be set slightly higher or slightly lower. 
The influx of a large number of people or the change of a technique 
would require a review of the goal established. 


The Economical Level 
It would be amiss for me to imply that we expect perfect 


work. We know our employees are going to make errors. We also know 
it would be far too expensive to try to eliminate all of their errors. 
In fact, when an activity reports e substantial reduction in errors 
and we ere convinced the results are honest, our first consideration 
is how much we can reduce inspection. It remains for top menagement 
to determine the economical level, after a consideration of cost and 


possible customer reaction. 


But what is more important and whet we ere anxious to pay 
for is the assurance of the constant protection which our quality 
control program affords. We want to know immediately when we should 


focus attention on results which ere not normal to a process. 


A further description of our quality control procedure may 
be desirable before going into actual applications. As we ell know, 
a quelity control chart in end of itself will not reduce errors. The 
remediable action taken efter data is obtained will be primarily 
responsible for any improvements. 


To get these data in good form st Aldens we use what we call 
a data sheet and an analysis sheet. A seperate data sheet is used 


10 


for every 100 units of work inspected as shown on pege 10. On this 
sheet we show a breakdown by operator and by type of error. The 100 
work units are a blend of a proportionate emount of work from each 
employee in the activity. For example, if there ere 10 employees 

we select spproximately 10 work units from each. The number of work 
units inspected on each employee is shown on the top half of the date 
sheet. On the bottom half we describe each error found and show the 
Operetor responsible. Also, the individual who made the mistake signs 
the sheet to indicate she was shown the error. We ere great believers 
that people like to see the mistekes which they make. This is much 


better than being told, "You're making too many errors...cut ‘em out!” 


The results shown on the individual dats sheets are then 
transposed to a weekly enslysis sheet which shows the totel number of 
work units inspected on each operstor for the week, together with a 
breakdown of the types of errors made, es shown on pege 11. We have 
found thet whet eppears to be an isolated type of error on any one 
day generally falls into a definite error category or pattern wher 


all errors are accumulated over en entire veek. 


The weekly analysis sheet is sent to the Operating Superin- 
tendent for review. A copy is reteined in the ectivity for study by 
the trainer end supervisors. So much for the mechanics of our progren; 


now to applications. 


First Applicetion 
Our first quality control application wes made in the 


spring of 1945 in one of the merchandise departments on the order- 
picking end checking operation. Order pickers walk or skate up end 
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Sheet No. 
ALDENS QUALITY CONTROL 
Data Sheet 


Date Inspector 
Units of Work Inspected 


When you complete the inspection of 100 work units, post the number of errors found on your control 
chart immediately. !f an error is serious or repetitive in character, call the matter to the attention of 
your supervisor at once. This worksheet to be circulated to and initialed by the following: 


Department Manager Supervisor Trainer 


FI053 


| 
Operator No. 
+ 
Description of Errors 
Operator 
Error No. Responsible 
10 -10- 


QUALITY CONTROL 


ANALYSIS SHEET 


Week Ending 


Total 


% Errors 


Entries shevid be mode on this recep sheet doily, with deta token from 
your Quelity Contre! work sheets of the dey previews. This report to be 
mode in duplicate ond the origina! sent te your Deportment Monoger 
each Mendey, for the week previews. 


| Cote 
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Gown stock eisles selecting merchandise by catslog number, sise, color 
end quantity. These work units are then taken to a conveyor line 
where another employee checks to see whether the selections ere 
correct, efter which the merchandise is placed on a conveyor belt 
which carries it to a spiral gravity chute dropping to the shipping 


room. 


The point of inspection is at the end of the conveyor belt 
eat the entrance to the grevity chute. This location is a neturel place 
to efford a random inspection. The errors found indicate the quality 
of work being performed by both the order picker and checker. 


Upon completing the examination.of 100 work units, the 
inspector goes to the control chart and posts the number of errors 
found in the sample. This result must be posted immediately, otherwise 
wach of the value of the program is lost. | 


Prior to the use of statistical quality control in this one 
merchandise department, we predetermined that e 3.00 per cent error 
retio existed. The process was within statisticel control but at an 
unsatisfactory level, as shown in the first half of Figure 1. As ea 
goal, we set an everage error ratio of 1.78 per cent, asking for a 
kO per cent improvement. Actually, the error ratio dropped within 
two weeks to 1.65 per cent, a 45 per cent improvement as shown in the 


second half of Figure 1. During the following months there was a further 


reduction to .70 per cent - - a quality level which was mainteined 
during subsequent months. This improvement from 3.00 per cent to 

-70 per cent wes ettributed to the enthusiastic support given to the 
new quality control progrem by the department manager, his supervisors 
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and employees. Special efforts were made to find errors, study 
errors, and to eliminate them at their source. The improvements made 
were verified by an independent check of outgoing packages. Adjustments 


with customers decreased accordingly. 


It is also interesting to observe that the production effic- 
fency of this department increased from about 82 to 107 during the 
period the quality performance improved. We believe that when errors 
are once brought under control and the results show a good performance 
is being maintained, there is a natural tendency to speed up production. 


A question often asked is why quality control should increase 
production. For the answer to this question we turn to member of the 
fair sex. While there ere not many young ladies working on quality 
control programs, there's one who hears a lot sbout mine and she can 
provide us with the answer to the question. 


Por several months after she learned to pull herself up to 
a standing position, my one year old daughter wouldn't let go. She 
was lacking in confidence. Once she acquired that confidence, sway 


she went. 


Confidence in anything we do encourages us to do more. 
Whether it's riding a bicycle, learning how to typewrite, or if it's 
operating a production line in a plant. When we have the assurance 
that quality is being mainteined at a satisfactory level, there is 
no reason to withhold production if greater speed is possible. 


After the success of this first installation, other mer- 
chandise department managers requested a similar program. It wes 
10 
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gradually expanded into all of the merchandise departments, the billing, 


shipping, exchange and returned goods departments. 


During the four year period beginning in January, 1946, and 
ending in December, 1949, the combined error ratios for all the mer- 
chandise departments was reduced by 57.9 per cent. It is believed 
that statistical quality control played « significant pert in bringing 
ebout this improvement. A few of the departments now heve an error 
retio as low as one tenth of 1 per cent, slthough our over-all plant 


objective is now one-half of 1 per cent. 


General Office Applications 
In the fall of 1945 the General Office Manager requested 


that statistical quality control be introduced in all of his departments, 
Clerical work performed in our General Offices includes the following; 


1. Open envelope, remove contents, verify remittance, apply 
cash impression to order blank. 


ree possibilities: Totel remittance incorrectly; 

apply wrong cas pression; fail to transpose customer's 

name end address from postal money order or personal check 
when information is missing from order blank and envelope, 


Read order to see whether any phase of transaction will not 
be handled in reguler mail order process. If so, apply 
special rubber stamps, make abstracts on special requests, 
inguiries and compleints. 


Soy ibilitie Pail to hendle special phase of 
ansaction. 


Record order on customer's index stencil; show date received 
and smount. Imprint stencil showing customer's name end 
address on shipping label and order. 


Pull entry "ticket" for eech catelog number ordered; circle 
size, color and quantity wanted. 


Error possibilities: Pull wrong tickets; circle wrong color, 
size or quantity. 


Schedule order for shipment. A machine operstion showing 
time, sisle, section, and bin number in Shipping room where 
order will be packed. 

Error possibilities: Switch tickets and send order to wrong 
customer. 


All of the above activities are under statistical quality 


control. Applications have also been made on key-punch operations 


in the General Offices and on freight and express routing activities, 
During the four complete years statistical quality control hes been 
in use in the General offices, 1946-1949, the combined error ratios 
have been reduced by 67.6 per cent. 


In one clerical department, the Order Reading, where ve 
originally had only one chart, we found we needed a still further 
breakdown by order classification, such as cash orders, no-cash orders, 
and credit orders. Accordingly, we now have charts on all of these 
separate activities. Also, in the General Office departments where 
we have both day and night shifts, we have a separate chart for each 
shift. As is generally expected, the night shifts started out with 
higher error ratios than the day shifts, but as the night force became 
more stabilized we now have better work in some activities then on the 
dey side. When each shift knows what the other is doing, competition 
becomes possible. The chart on the wall for each shift speaks for 
itself. 


About a year and a half ago statistical quality control was 
introduced into the Credit Department, where charts sre now maintained 
on posting-checking operations, credit approval, follow-up typing, 
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files and related ectivities. Comparing January, February end March, 
1950, with the same three months last year, combined error ratios for 


all credit department activities have been reduced 37 per cent. 


The manner in which data is obtained for the control chart 
on the filing operation may be interesting. Before signout of work 
is made to file clerks duplicate stencil impressions showing customer's 
neme and address are removed from the papers. After 100 nemes have 
been accumulated and the papers filed, a lookup is made to see whether 
they were filed correctly. 


Monthly Quality Control Meetings 
Each month we hold a quality control meeting in which all 


of the charts for the previous month are reviewed. This meeting lests 
about one hour end is attended by 211 department managers, the plant 

superintendent, end the Director of Mail Order operations. All of the 
charts are reviewed and department managers ere called upon to discuss 


any unusual results, whether unusually good or unusually bed. Supple- 


mental efforts to go "beyond the chert” are described by manegers who 


have found new devices helpful. 


A definite amount of pride and assurance results when a 
manager with good results knows his charts are going to be seen by top 
management at this monthly meeting. Also, at this time, each manager 
has the opportunity to see all the other charts and heer discussion of 
then. 


This raises the question of the assignment of responsibility 
for quality control. In our plant we feel that the control of quality 
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is just es much 4 manageriel function and responsibility as the control 
of production, unit cost, absenteeism, or any other administrative 
control. For this reason, each of our department managers acts as his 
own quality control manager. He is personally responsible for the 
Quality of work his people turn out. To take away this responsibility 
and transfer it to the Training Department or to the Inspection Depart- 
ment would be to relieve the manager of one of his most importent 


functions as a manager, namely, to turn out quality production. 


Beyond The Chart 


The question of how far the manager should go to accomplish 
@ necessary improvement is left to his discretion. Presumably, he knows 
his people best, and takes the action he deems necessary under the 
circumstances. Such questions as to the publicity which should be 
given to individual error makers, the penalties which should be imposed 
for poor work, the special awards which should be given for good work, 


ere left entirely to his judgment. 


Some managers attach the individual data sheets to the bottom 
of the control chart. Other managers have individual error slips made 
out, signed by the employee responsible, then posted on the control 
chart. Another places a gold star besides an operator's name on a 


bulletin board for perfect work done during an entire week. Usually, 


a plant-wide program has to be given a "booster" shot from time to time 


and for this we depend upon the originality of the managers working 
in the prograz. 


The department manager must be constantly searching for weys 


to eliminate error hazerds. We know of no better way to reduce errors 
than to eliminate them st their source. In clerical operations, 
particularly, we have found that more opportunities exist than one 
realizes, For example, en error hazard is always present when figures 
ere transposed, In our Exchenge Department the practice had been to 
carry forverd a "total settlement" figure from the spece on the beck 
of the returned goods billing jacket to the flap portion on the front 
of the papers. The Refund Department would then issue a bank check 

to correspond with the emount entered on the flap. This type of error 
ween't numerous but the hazard wes alweys present and an error was mede 
occasionally. Rather then risk an error in transition, the Refund 
Department now works directly from the "totel settlement" figure, 
entirely eliminating the error hazerd at its source. 


In order filling operations we have found that proper lighting, 
good bin numbering, end improved bin-teg color combinations eliminate 
error hazerds. To reiterate, when ean error condition exists, attention 
should first be directed towards completely eliminating the error 
hazard, If this cannot be accomplished, then the second consideration 
should be how to reduce the error ratio. 


Say It In English 
When symbols or abbreviations are used in plent operations, 


@ good practice is to make them self-explanatory. Symbols or signels 
may work all right with experienced employees, but they needlessly 


complicate the training of new people. To illustrate, for years "Xxx" 


on our printed entry ticket meant "time payment order." The word 
"credit" or "T.P." do not need any interpretation. A pink slip inserted 
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vertically to extend below the jacket flap meant "Send customer ea 
catalog.” "P.P." meant "Postpaid." In training new employees it is 
much better to have expressions which say exectly what they mean. 


There is only one activity in our entire plent on which we 
use X end R. This is in the measurement of yard goods. Most of our 
piece goods are stocked in bolts and sold by the yard, the sales 
running into many thousands of yards daily. The customer may purchase 
any quantity desired but not less than 1/8 yard. Each order is cut to 
measure. Although we would like to cut the exact length, we find the 


natural tendency is to over-cut. 


Prior to the introduction of statistical quality control on 
the cutting operation, an accumulation of samples indicated the averege 
over-cut was 1.97 inches per cut, with both the everage and range very 
erratic. The process was not in statistical control. We found that 
the order picker with the highest operating efficiency and the one 
thought to be the best of 20 such employees was giving away, occasionally, 


as much as 134 yards of materiel an hour. 


We found two principal causes for over-cuts...uneven tears 


and too many thumbs. By concentrating on both these factors we have 


been able to reduce the average over-cuts to 1.14 inches with further 
reductions indicated. Also, the process is now consistently within 
statistical control. A reduction of one inch per cut in the yard goods 
department is equivalent to a savings of between $15,000 and $20,000 


per year. 
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Samples are taken in lots of 5 pieces from each operator, at 
rendom throughout the dey. The samples are then messured for exact 
length. The average of the over-cuts found in the sample is plotted 
on an average, X, chart; and the variation of the over-cuts found in 
the sample is plotted on a range, R, chart. Each plotting is identified 
with the name of the employee responsible. 


A Five Year Eveluation 

After five years of application in our plant, stetisticel 
quality control is now considered an integrated part of our standerd 
opersting procedures, ss is time and motion study, plant ley-out, job 
evalustion, or any other management control. The appraisal of the 
value of the methods of our depertment managers is best illustrated 
by one whose quality of production is so high there is rarely an error 
plotting on his control chart. Yet, he refuses to discontinue the 
procedure. He states that "The chart is the best advertisement I 


have." 


To summerize whet we believe to be the outstending sdvan- 
teges of the methods: 
1, he epproach is anslytical. The randomness of the sample 
tee of variations in the sample end e point 
out of control, the classification of the deta by employee 


and type of error, end the psychologicsal effectiveness of 
the chart all add up to the best known scientific epproach. 


Variation is ee ee Whether or not we cere to admit 

at variations w occur we know they exist. Employees 
and managers elike feel better about a production control 
when they know the results expected will be reasonsble and 
uniformly applied. 


hes confidence. The essurance thet quality is 
eing constantly maintained in the process builds confidence 
and permits increased production. 
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sources. 


need for aK cortrol is ever present. The control 
t is ere er that the control quality is an ever 
present sretien, not one which can receive only occasional 
attention. 


fet results become productive. Inspection results 
whic ormeriy went into a des aver sre now out in the 
open, on the control chart, where everyone can see what is 


taking place. A representetive of management can tell et 
@ glence: 


Whether inspection is being made daily. 

How much inspection is being made. 

The number of errors found in each 100 work units. 
The trend errors are taking. 


Employee reaction is favorable. The employee knows whet is 
pags, of him and how he 1s doing while his work is being 
done. 

A word ebdout the sampling of incoming merchandise from supply 


A maenufecturer cannot expect his distributor to examine every 


piece of merchandise delivered to him, nor can the distributor afford 
to do this. How much better it would be if a quality control chert 


could accompany esch shipment of merchandise, showing the quality which 


was maintained during the manufacturing process! This would reduce 


or entirely eliminate the need for sampling inspection by the distri- 


butor, @ savings which could be passed on to the customer in the form 


of ea lower price. 


In my opinion, the day is not far off when purchese orders 


will read "Quality Control process chart must accompeny merchendise." 
Manufecturers will proudly describe their products as "Produced under 
quality control.” Everybody will know whet everybody else is telking 
ebout and everyone will know whet to expect. 


WHanager of Inspection, Aldens, Inc.; Fellow, Americen Society for 
Quality Control, State University of Iowa Section. 


10 


-21- 


5. 
5 
a. 
7 Cc. 
| 
: 


